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ABSTRACT 



This document contains the Conference Agenda, followed by 
highlights from 42 Spotlight School systems across the country. The schools 
briefly share their experiences, evaluation techniques, and tell their 
success stories about the positive impacts technology is having on teaching 
and learning. Conference research materials are then presented. Papers 
include: "Perspectives on Technology and Education Research: Lessons from the 
Past and Present" (Margaret Honey, Katherine McMillan, and Fred Carrigg) ; 
"Convergent Analysis: A Method for Extracting the Value from Research Studies 
on Technology in Education" (Cathleen Norris, Jennifer Smolka, and Elliot 
Soloway); "Observing Classroom Processes in Project-Based Learning Using 
Multimedia: A Tool for Evaluators" (William R. Penuel, and Barbara Means); 
"The Technology/Content Dilemma" (Shelley Goldman, Karen Cole, and Christina 
Syer) ; "Technology: How Do We Know It Works?" (Eva L. Baker) ; "Documenting 
the Effects of Instructional Technology: A Fly-Over of Policy Questions" 

(Dale Mann); "The Cyberspace Regionalization Project: Simultaneously Bridging 
the Digital and Racial Divide" (Jonathan D. Becker) ; "New Directions in the 
Evaluation of the Effectiveness of Educational Technology (Walter F. 

Heinecke, Laura Blasi, Natalie Milman, and Lisa Washington); "Measurement 
Issues with Instructional and Home Learning Technologies" (Charol 
Shakeshaft) ; "The Idaho Technology Initiative: An Accountability Report to 
the Idaho Legislature on the Effects of Monies Spent through the Idaho 
Council for Technology in Learning" (State Division of Vocational Education, 
State Department of Education, Bureau of Technology Services) ; "West Virginia 
Story: Achievement Gains from a Statewide Comprehensive Instructional 
Technology Program" (Dale Mann, Charol Shakeshaft, Jonathan Becker, and 
Robert Kottkamp) ; "Testing on Computers: A Follow-Up Study Comparing 
Performance on Computer and on Paper" (Michael Russell); "Testing Writing on 
Computers : An Experiment Comparing Student Performance on Tests Conducted via 
Computer and via Paper-and-Pencil" (Michael Russell and Walt Haney) ; and 
"Critical Issues in Evaluating the Effectiveness of Technology" (Mary McNabb, 
Mark Hawke s , and Ullik Rouk) . A short profile of U.S. Secretary of Education 
Richard W. Riley is then given. This is followed by an outline of the 
education and professional experience of Paulo Renato Souza, Minster of 
Brazil and a brief discussion on the implementation of the Brazil -US 
partnership in education. (AEF) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 
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Washington Court Hotel 
July 1 2th and 1 3th, 1 999 - 
Washington, D.C. 

The Summary of the 
Conference is now ini 

In addition to basic text, 

the summary is also available in 
portable document format 
(PDF). In order to read PDF 
documents, you will need to 
have an Acrobat Reader 
installed on your computer. For 
information about how to 
download the Acrobat Reader 
free of charge, please visit the Adobe web site . 

Please note that the PDF files may include forms and exhibits omitted 
from the text version. 
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Agenda 

The Secretary's National Conference on 
Educational Technology: 
Evaluating the Effectiveness of Technology 



July 12-13, 1999 




July 12 th - Monday 
Day One 

9:00 am-10:00 am OPENING SESSION GRAND BALLROOM 

Multi-media presentation — "Virtual School Visit" 

Greetings — Linda Roberts, Director, Office of Educational 
Technology, U.S. Department of Education 

Secretary’s Address — Richard W. Riley, Secretary of 
Education, U.S. Department of Education — Introduction by 
Phil Bigler — 1998 National Teacher of the Year 

Conference Orientation — Diane S. Reed, Technology 
Teacher in Residence, U.S. Department of Education 
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Conference 

1Q:00 am- 1.1:15 am PLENAR Y SESSIONS GRAND BALLROOM 

"Statewide Technology Evaluations" 

Gordon Ambach Executive Director, Council of Chief State 
School Officers — Moderator 

"West Virginia’s Basic Skills / Computer Education 
Program: An Analysis of Student Achievement" 

Henery Marockie, State Superintendent of Schools, West 

Virginia Department of Education 

Lewis Solomon, Vice President, The Milken Exchange 

"The Idaho Technology Initiative: An Accountability 
Report to the Idaho Legislature" 

Mike Rush and Cliff Green, State Division of Professional 
Technical Studies, Idaho Department of Education 

Question & Answer Session — Gordon Ambach 

Closing Remarks — Senator Jay D. Rockefeller, WV 

12:00 pm-l:30 pm LUNCHEON GRAND BALLROOM 

"Virtual School Visit" 

Luncheon Address — Paulo Renato Souza, Minister of 
Education, Brazil — Introduction by Terry Peterson, Counsel 
to the Secretary, U.S. Department of Education 

1:30 pm-3:30 pm SPOTLIGHT BREAKOUT/ WORKING SESSIONS 

"Setting the Context and identifying successes and 
barriers" 

Spotlight Schools will be grouped with other schools and 
researcher/e valuators and facilitators to discuss findings from 
their evaluations. Schools will share their experiences, 
evaluation techniques, and tell their untold success stories 
about the positive impacts technology is having on teaching 
and learning. Participants will gain insight into what is 
happening in our schools across the country and discuss 
findings that have not been captured by the press or research 
community. There are 9 breakout groups (refer to notebook 
for rooms and groups). General invitees are encouraged to 
visit the breakouts, and participate in the discussions. 




3:30 pm 



BREAK GRAND BALLROOM FOYER 
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Conference Agenda 



4;00 pm-5:30 pm 


SPOTLIGHT BREAKOUT/WORKING SESSIONS II 

"Identify learning criteria and accessing the impact of 
technology" 

Schools and facilitators will go to .he same breakout room as 
session 


6:00 pm-7:30 pm 


RECEPTION AND SPOTLIGHT SCHOOL SHOWCASE 
A TRIUM BALLROOM 

Members of Congress invited. Spotlight Schools will be 
available to discuss their technology projects. 


7:00 am-8:45 am 


July 13 th - Tuesday 
Day Two 

CONTINENTAL BREAKFAST ATRIUM BALLROOM 
Spotlight School Showcase Continues 


9:00 am 

\ 


PLENAR Y SESSION GRAND BALLROOM 

"What ar 2 we learning and what do we need to learn 
about technology effectiveness and impact?" 

A panel discussion with the following leading researchers and 
evaluators: 

Dale Mann, Interactive Inc. — Moderator 

Eva Baker, Center for Research and Evaluation, 
CRESST/UCLA 

Margaret Honey, EDC/Center for Research and Evaluation 
Charol Shakeshaft, Interactive, Inc. 

Elliott Soloway, University of Michigan 

Question & Answer Session 


9:45 am-10:30 am 


BREAK 
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10:30 am-ll:15 am PLENARY SESSION — " The Media, the Stories, the 

Impact" 

National, regional and local perspectives from members of the 
media and their experience covering educational technology 
issues. 

Judy Salpeter, Editor-in-Chief 

Technology & Learning Magazine 

Andrew Trotter, Education Week 

Alan Duke, Managing Editor CNN Student Bureau 



Moderated by : 

Dennis Gooier, Assistant Director, NCREL 

Jayne James, Kansas Department of Education — Moderator 

12:00 pm-l:30 pm LUNCHEON GRAND BALLROOM 

’’Virtual School Visit” 

Lucheon Speakers: 

Congressman William F. Goodling, PA 
Chairman of Education and Work Force Committee 
Congressman Michael N. Castle, DE 



1:30 pm-2:45 pm SPOTLIGHT BREAKOUTS/WORKING SESSIONS II 

”How to implement evaluative criteria” 

Breakout sessions continued in same rooms 

2:45 pm-3:00 pm BREAK GRAND BALLROOM FOYER 

3:00 pm-5:00 pm CLOSING SESSION GRAND BALLROOM 

’’What did we find? What did we learn? Where do we go 
from here?” 

Panelists: 

Eva Baker, UCLA 

David Dwyer, Consultant 

Kathleen Fulton, University of Maryland, 

Margaret Honey, Center for Technology and 
Children 

Dale Mann, Interactive, Inc., Columbia 
University 

Robert McNergney, University of Virginia 



O 

g j^£p://www. 



ed .gov/T echnology/T echConf/1 999/agenda, 



c 

html (4 of 5) [4/24/2001 2:26:54 PM] 



Conference Agenda 



Lynn Schrum, University of Georgia, ISTE, 
President 

Elliott Soloway, University of Michigan, 
Walter Heinecke, University of Virginia 

Linda G. Roberts, Director, Office of 
Educational Technology — Moderator 
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potligh? Schools 



Ainsworth Community Schools. NE 



Anderson Countv Schools. TN 



Aneth Comm unity School. UT 



Blackfoot School District, ID 



Boston Public Schools, MA 



Bristol Warren Regional School District, Rl 



Carnoton Elementary School. NH 



Carteret County Public Schools, NC 



Cherokee Countv School District, AL 



Cherry Creek Schools, GO 



Chicago Pub lic Schools, IL 



Drew School District, MS 



East Detroit Public SchQQ&M 



Edaewood Elementary School. GA 



Evergreen School District. WA 



Fabens School District. TX 



Gold Oak Union School District, CA 



Helena School District. MT 



Henrico Countv Public Schools. VA 



Hope Valiev Elementary School. Rl 



Hundred High School, WV 



Jennings School District. MO_ 



Kaventa Unified Schoo l District. A Z 



Kenton Countv Public S chools. KY 



Kuna School District, ID 



Lennox School District. CA 



Little Falls Co mmunity Schools. MN 



Mantua Elementary School, VA 



Montgomery Countv P ublic Schools. MD 
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• Middletown Countv Schools. OH 

• Milwaukee Public Schools. Wl 

• New Haven Public Schools. CT 

• Norman Public Schools. OK 

• Ohio Countv Schools. WV 

• Okaloosa School District. FL 

• Oswego School District. NY 

• Richland School District #1 . SC 

• Rov Municipal Schools. NM 

• SAP. #4. ME 

• Sherman Elementary School. WV 

• South Harrison Community Schools. IN 

• North High School. KS 
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Ainsworth Community Schools, NE 

The Secretary's Conference on Educational Technology-1999 



Ainsworth Community Schools 

Nebraska 

Ainsworth, Nebraska, has been nicknamed the "Middle of Nowhere", and with good reason. Ainsworth 
lies in the sparsely populated Sandhills of north central Nebraska, approximately 200 miles from Omaha 
and 420 miles from Denver. A community of 1,870 members, Ainsworth serves as the county seat of 
Brown County, which has an area of 1,221 square miles and supports a total of population of 3,657. 
Ainsworth has a rural, agricultural economy, with major economic activities being farming, ranching, 
and cattle feeding. Brown County supports seven Class 1 "country" schools, as well as the Ainsworth 
Community Schools system. 

Ainsworth Community Schools consist of Pleasant Hill Elementary, McAndrew Elementary, Ainsworth 
Middle, and Ainsworth High Schools. The school system serves 668 students, the vast majority of which 
are Caucasian. In order to provide students with educational experiences that include technology, the 
Ainsworth Community Schools incorporated technology as a component of its School Improvement Plan 
in 1994. A technology committee was appointed to enhance the use of technology to improve students' 
communication and critical thinking/problem solving skills. The district was aided in 1995 by an 
Excellence-in-Education Grant, a grant designed to expand the use of technology in the classroom. In 
1996, Ainsworth Community Schools was selected to be a participant in the federal Challenge Grant. 
This grant's intention is to enhance curriculum integration with the use of technology. Through the 
Challenge Grant's Connection Project, Ainsworth has been able to provide its children with 
state-of-the-art technology instruction and equipment in spite of its geographical isolation. 

-###- 




TAnderson County Schools! 



JL 



0 



r? 



://www.ed.gov/Technology/TechConf/1999/profiles/ainsworth.html [4/24/2001 2:27:02 PM] 



O 



Anderson County Schools, TN T?o" 1 r: “ I 

The Secretary's Conference on Educational Technology-1999 



Anderson County Schools 

Tennessee 



Anderson County Schools 
HOMEPAGE 

In virtually every state in the nation, reform efforts are dramatically raising expectations for students, and 
consequently, for teachers. In response to these reform initiatives, educators are being asked to master 
new skills and make necessary changes in their classrooms. To meet these new expectations, educators 
must deepen their content knowledge and acquire new methods of instruction. They need more time to 
work with colleagues, to critically analyze the new standards being suggested, and to revise curriculum. 
Educators need opportunities to develop, master and reflect on new approaches to working with students. 
All of these activities fall under the general heading of professional development. 

A key lesson learned about school reform from the past decade is that far more time is necessary for staff 
learning and planning than is currently being made available. Staff development days, typically 
workshops, and brief meetings before, during, or after the school day when other responsibilities tug at 
the participants are grossly insufficient for the profound learning and planning which are essential to 
successful improvement in teaching and learning. 

The importance and placement of professional activities will require the support of all stakeholders, 
including parents, students, and community members. Before redesigning professional development 
activities, it is important to understand the research on best practices in professional development. 
Research clearly defines the following assumptions: 

Ongoing professional development is required if it is to result in significant change. School change is the 
result of both individual and organizational development. The goal of professional development is to 
support the inquiry into the study teaching and learning. Teachers learn as a result of training, practice, 
and feedback, as well as individual reflection and group inquiry into their practice. Professional 
development is essential to school development. Professional development should be primarily school 
focused and embedded in the job. 

Professional development programs based on these beliefs are quite different from those based on 
traditional assumptions. While district wide workshops still will be appropriate on occasion, most 
professional development should be school based. Educators should attend hands on workshops and 
conferences, and be involved in a variety of ongoing, job embedded learning activities, such as study 
groups, action research, peer coaching, curriculum development, and case discussions. 

In Anderson County In Anderson County, the Office of Technology developed a system wide technology 
plan, which allowed school sites to develop their technology plans. After evaluating school, teacher, and 
student needs in the school technology plans, professional development activities can be developed. 
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System Technology Plan (Handout #1, Sections 1 A3) Because KAl2 education has traditionally been 
restricted by limited budget funds, it was imperative that whatever funds were expended on technology 
be made to serve the most students possible. This has forced most purchases to focus on meeting today's 
needs only. Unfortunately, as changes in technology come more rapidly and software becomes more 
demanding, technology bought with only today in mind become obsolete very quickly. 
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Aneth Community School 

Bureau of Indian Affairs 
Utah 

Aneth Community School is located in a remote region of Southeastern Utah near the Four Comer 
National Monument. The community is within the boundaries of the Navajo Nation Reservation. The 
School was built is 1965 with one education building, two dormitory buildings, one kitchen/dining 
building and a small maintenance building. In the tradition of the time, multiple single family and 
apartment style dwellings were built adjacent to the school facility. Over the years there have been few 
changes except for the change from K-8 to K-6 and the demolition of one of the dormitory buildings. 
Until the Summer of 1995 technology at the school consisted of a few computers scattered around in 
classrooms and offices and a modest number of telephones in administrative offices. 

In the Summer of 1995 Aneth Community School was included in a Star Schools Grant as a minor 
partner for which a new computer lab and a distance learning receive site was installed on site. The key 
feature of this installation was the microwave tower that provided for full motion video, voice and data to 
come and go from the school. The two way video/audio connection was used from the start, but, the data 
connection was not utilized until later. This partnership had shown the need for more technology at the 
site and the local administration began working on a plan to write a Technology Literacy Challenge Fund 
Grant and dedicate base funding to the furtherance of technology. In the Summer of 1997 the Office of 
Indian Education Programs awarded a five year grant to Aneth Community School. That same year the 
Principal decided to make a large one time infusion of resources into technology. 

The Summer and Fall of 1997 was a frenetic time at Aneth Community School. First to come was an 
extensive wiring project which entailed data, voice and video cabling being pulled in all the buildings on 
campus and being tied together by a fiber optic backbone. Shortly after the new year we received 
forty-four new computers and a new server. One new computer was put into every classroom for teacher 
use along with administrative offices. Additionally, computer mini-labs were established in the 
kindergarten to third grade and color printers were placed in every classroom. Through the Spring and 
Summer of 1998 we selected 27 inch televisions with internal scan converters and placed them into each 
classroom also. 

With the campus cabled and computers, printers and display devices into each classroom attention was 
focused on campus communications and faculty development. Major projects in the Summer of 1998 
included the installation of a video head end system, a telephone system complete with phones in every 
classroom, voice mail and a digital satellite downlink which brought eight bachelors and four masters 
degrees to Aneth via distance learning technologies. 

All along we have kept an eye on how quickly the faculty and staff will assimilate new hardware and 
software. Ample opportunities have been provided in small clinics on Fridays and during week long 
session held once or twice a year. Attention has always been placed upon creating an atmosphere for 
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success by the learner. 



The future looks bright as we head into our third year of funding. The foundation is solid as we look to 
turn the comer and attempt to create an environment where computers and pencils are equal in their 
appropriate tasks and classroom teachers become coaches and facilitators of all the learning tools at their 
command. Students, parents, faculty, school board members and administrators all agree that there is a 
special excitement at Aneth Community School that has not been here for along time. 
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Blackfoot School District 

Idaho 



Blackfoot School District 
HOMEPAGE 

Over the past five years Blackfoot School District has placed a stronger emphasis on the integration of 
technology into the classroom to positively impact student achievement. Assistance from the Idaho State 
Department of Education in conjunction with Federal and a variety of grant initiatives (Technology 
Innovation Challenge Grant, Technology Literacy Challenge Fund, Albertsons Waterford Initiative and 
ICTL Funds) have enabled the district to move toward implementation of the District's Technology Plan. 

The Blackfoot School District Technology Plan is based upon a comprehensive needs analysis that 
addresses student needs, teacher needs, curriculum integration, and the availability of technology to 
accomplish stated goals. 

Technology Action Plan 

In an effort to positively impact student achievement by leveraging the potential of technology, a 
three-year, educationally driven plan has been developed with both intermediate and long term goals 
established. Based on research in educational technology, the plan addresses the need to consider 
professional development, the availability of technology and curriculum integration. The anticipated 
academic outcomes are based on a district academic audit, teacher and administrative perceived needs, 
academic testing and performance indicators, and parental input. The identified academic areas of focus 
are language arts and math. It is recognized that various elements must be in place to achieve the desired 
outcome of increased student achievement. These elements include: 

Professional Development 

Assisting teachers to gain the necessary skills to optimize technology. The effective implementation of 
technology in the classroom requires not only the technical skills to operate the technology, but changes 
in classroom management and organization must also take place. This requires an investment in time and 
training (Idaho State Goals 1, and 3). In the process, teachers will move toward the State requirements 
for technology certification. 

Administrative training which promotes the effective integration, evaluation and administrative uses of 
technology (Idaho State Goals 1, 2 and 3). 

Availability of Technology 

Provide teachers and administrators with the tools to be successful. To attain this goal, a three-year plan 
has been developed which places computers in the classrooms in accordance with the State suggested 
ratio and research of 5 computers per classroom (State Goal 2, 5 and 1). In order to impact student 
achievements they must have access and adequate time on task. It is not fiscally possible to equip all 
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classrooms at one time therefore a staggered schedule which enables the disbursement of computers by 
gr-ade level starting at the third grade and progressing through the twelfth grade has been established. 

This promotes adequate access to enable time on task and the ability to train teachers on a district-wide 
basis by grade level to have the skills and knowledge to benefit from technology. It is the individual 
school's responsibility to provide printers, consumables and Internet access at the classroom level 
through the use of building funds. Internet connectivity is available to academic classrooms in all 
buildings except the sixth grade center where there is currently a lab online and it is anticipated that 
individual rooms will be connected by the end of the 1998-1999 school year. Building maintenance will 
provide the remodeling, upkeep or physical requirements to accommodate the acquisition and 
maintenance of technology. 

Provide technical support structures minimizing frustration and promoting effective use. The District is 
expanding their support through the use of in- district support and student training (State Goals 2, 5,7 and 
8 ). 

Curriculum Integration 

Purchase and implementation of well researched software addressing the identified academic outcomes 
(State Goals 1, and 4). Supportive training which promotes the effective use will be provided. 



National Standards for Technology in Teacher Preparation have been used as a guideline in developing 
effective staff development. Involvement of teachers in the integration of technology into the classroom 
frequently involves incentives. The District is providing inservice aimed at using technology effectively 
to enhance academic achievement and promote the effective administration of schools at no cost to 
participants. Necessary tools to be successful, (training, software and equipment) will be made available. 
In addition, release time is available for a wide variety of training opportunities. 

-###- 
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Boston Public Schools 

Massachusetts 



Boston Public Schools 
HOMEPAGE 



Boston Public Schools (BPS) has received many awards for its leadership role in technology among large 
urban cities, Boston's plan LINC Boston (Learning and Information Network for the Community) is a 
comprehensive five year plan to address all of the key elements involved in implementing a substantive 
technology program in our schools. As of Spring 1999, in the third year of the plan, Boston can attest to 
the following accomplishments and immediate goals. 

Networked Schools 

Every one of Boston's 130 schools has a "starter network" (computer lab, library, principal's office, and 
4-8 classrooms) connected to a wide area network based at Court Street, our central administrative office. 
Eleven schools are completely networked, with plans to complete the networking of all schools by 2002. 
Boston is the first major urban school district in the country to have networks and high-speed Internet 
access throughout every school. 

Hardware and Software 

In June 1995, Boston had a 1:63 computer student ratio for new computers. By June 1999, we will have 
installed 10,500 additional state of the art computers, bringing Boston to a 1:6 computer to student ratio. 
This accomplishment puts Boston well on the way to reaching the goal of one computer for every four 
students and a computer for each teacher, a commitment made by Mayor Menino in his State of the City 
Address in January 1996. All computers come "loaded" with software; Microsoft Office on all machines, 
and in addition, Claris Works, KidPix, and HyperStudio on all elementary and middle school equipment. 
Each school also receives a budget to purchase additional curriculum software. 

Assistive Technology 

Boston Public Schools, supported by funding from city, state and federal funds, has made a major 
commitment to providing computers and appropriate assistive technology to all special needs classrooms 
by 2002, as teachers participate in professional development to understand how to use these technologies 
to support student learning. 

Professional Development 

Boston has developed Technology Competencies to be achieved by all staff. These competencies, based 
on those recognized by the International Society for Technology in Education (ISTE), and supported by 
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the Boston Teachers Union, have five levels of proficiency. At each level, BPS offers free on-site courses 
to teachers, principals, and other staff to help them achieve those competencies. In addition, teachers are 
awarded computers for their classrooms, once they have completed competencies at each level, and have 
been "coached" by one of their colleagues to produce technology based materials for their classrooms. 

By June 1999, 2,000 teachers (40% of all Boston teachers) will have received their first computer and 
printer, recognizing their development of "productivity tools" for their classrooms. 400 teachers will have 
participated in curriculum integration projects and received additional classroom computers. Beginning 
in 1999, new teachers must document technology competency at the Novice level or participate in 
summer workshops prior to beginning teaching in September. New teachers will be eligible to participate 
in coaching during their first year of teaching and to receive technology for their classrooms. 

Student Competencies 

In September 1998, a team of Lead Teachers representing all grade levels and including subject area 
teachers, bilingual, SPED, and computer instructors, began to work to develop a set of Student 
Technology Competencies. (Being sent with Technology Plan) They based their work on the standards 
recommended and recognized nationally by the International Society for Technology in Education. In 
presentations to groups of teachers, principals, parents, and the Leadership Team, the Competencies have 
been well received. Teachers and principals alike appreciate the fact that clear and consistent 
expectations are established for students and teachers at all grade levels and that the Competencies are 
directly connected to the City-wide Learning Standards. 

It is important to note that these Competencies cover the use of technology as a tool to enhance learning 
in all subject areas. Technology education, defined as a study of the machinery that each generation 
develops to make society's work easier and more productive, is covered both at the state level and in 
Boston, as part of the Science curriculum. 

Curriculum Integration 

Boston Public Schools, supported by grants from the federal and state governments, as well as by IBM, is 
developing web-based resources to support the dissemination of exemplary curriculum materials which 
support Boston's Citywide Learning Standards, as well as on-line rubrics to support the assessment of 
student work. Hundreds of teachers participate in technology based curriculum workshops and coaching 
to share the development of best teaching practices. 

Libraries 

All Boston Public High School libraries, as well as 10 elementary and middle school libraries are 
automated, in a unique partnership with the Boston Public Library. All students in these schools have 
access to all of the resources of the Boston Public Library, directly from their schools. They use Boston 
Public Library cards to check out books at their own schools, as well as being able to look up books at 
every branch library, and reserve books and have them delivered right to their schools. This partnership 
between school and public libraries is the first of its kind in the country. 

Support 

The greatest challenge for all school systems developing technology programs is providing adequate 
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support for the technology. To address this concern, Boston has worked on several fronts, including the 
development of a remote management system for all its networks, and implementation of a sophisticated 
Help Desk system which responds to many problems over the phone and deploys teams of technicians to 
resolve others. Boston anticipates that a major source of support for its technology will be its own BPS 
students. Though partnerships with Microsoft, 3Com and Cisco and other technology companies, 
Boston's Offices of School-to-Career and Instructional Technology have developed courses and 
apprenticeships for students ranging from A+ computer repair to networking, and systems operation. 
After school and during vacations, students work as apprentices to BPS technicians, supporting the 
technology in schools. BPS is developing a 13^* and 14^ year program in collaboration with CityYear 
and Americorps for our graduates to continue their work as stipended interns in the schools, and at the 
same time continuing their technical training. 

Partnerships 

Boston Public Schools has received tremendous support for its technology programs. The LINC Plan has 
raised more than $38.75 million dollars, including $1 5.75 from private partners, and $23 million in 
grants. Major business partners include 3Com, Microsoft, HiQ, Intel, Bell Atlantic, Boston Edison, and 
more than 100 other companies. 

Awards: 

The LINC Boston Plan has won the following awards: 

• National League of Cities 1998 Innovations Award 

• Massachusetts Software Council 1998 Innovator Award 

• John Hancock 1998 Innovations in Education Award 

• Macy 1996 Leading Boston into the 21 st Century Award 

• Nominee for Smithsonian Institution 1999 Innovation Award 
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Rhode Island 

Bristol Warren Regional School District initiated the implementation of a district wide technology 
system necessary to improve teaching and learning in 1997. This initiative also supports professional 
development, changes school culture and addresses individual student needs. The district technology plan 
includes the systematic purchase of hardware and software. Ongoing professional development is 
provided for our staff. The district attempts to adequately budget for the resources necessary to sustain 
district technology initiatives. Kickemuit Middle School, through an extensive renovation project, has 
become our first state of the art technology enhanced learning environment. 

Bristol Warren Regional School District is implementing a wide area network that will provide the 
district with technology systems necessary to achieve the reforms listed above. We began this effort in 
the summer of 1997 and plan to have the system fully operational by fall 1999. Implementing the district 
wide area network has required wiring each building, installing hardware such as NT servers, purchasing 
district software and creating a district frame relay cloud with T1 Internet access. 

To ensure the success of our technology implementation, we have planned a variety of professional 
development opportunities for teachers. This professional development is a combination of state and 
local efforts. Bristol Warren Regional School District provides workshops for all teachers. These 
workshop focus on specific technology skills as well as curriculum integration. 

Fifty teachers in Bristol Warren have benefited from the RI Foundation Teachers in Technology 
Initiative (RITTI). These teachers have received a laptop computer and two weeks of technology 
training. Upon returning to the district, a number of them with the proper district support, have 
significantly improved the technology integration in their classroom and professional endeavors. An 
additional thirty-two teachers will participate in the RITTI program this summer. 

The Kickemuit Middle School renovation provided the opportunity to integrate technology into all 
aspects of the middle school. The school has an extensive network with multiple high speed Internet 
connections in every classroom. The multimedia library is automated and includes numerous student 
workstations. Every teacher has a desktop workstation with projection capabilities. The school also 
includes three computer labs - a teaching lab, a sending lab and a writing lab. 

Professional development at Kickemuit is provided using various training models. Every teacher 
attended a basic training workshop prior to implementation in 1998. One of the most successful models 
presently used is "just-in-time" training sessions. The school technology facilitator is responsible for 
providing ongoing technology curriculum integration training opportunities and workshops. Team 
planning periods focus on specific training and curriculum needs. Site-based workshops focus on 
identified technology needs of the school. Support is also provided electronically through e-mail, 
listservs and web pages. 
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Using Kickemuit Middle School as a model for educational technology, Bristol Warren Regional School 
District's goal is to expand technology in all schools throughout the district. We look forward to 
improving student learning through increased technology integration district-wide. 
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Campton Elementary School 

New Hampshire 

Our technology implementation plan provides the vehicle to revolutionize the way we teach and the way 
students leam. Skillful use of technology supports the development of process skills such as flexibility, 
adaptability, critical thinking, problem solving, and collaboration which are essential to success in our 
rapidly changing information age. The student and faculty at Campton Elementary School are able to 
access information, manipulate data, synthesize concepts and creatively express ideas using video, text, 
and audio media. Information technology provides a depth richness of instructional approaches to reach 
student of all learning modalities. It allows educators to better serve the diverse learning styles of our 
students and educate them through a spectrum of multiple intelligences. The integration of technology 
across the curriculum provides access to otherwise unattainable information and better prepares our 
students for a life of learning. 

Our curriculum is ambitious. Successful implementation of our information technology vision requires a 
tremendous commitment and, because we truly believe in maintaining a high quality educational 
program, we provide an ongoing monthly program of inservice training. Therefore, each student, teacher, 
staff member, and administrator much regular access to the technologies that enables him/her to keep 
up-to-date with technology. 

It is our school vision that all members of our community will access, manipulate, integrate and 
communicate information within, and beyond, the school setting. Electronic links (school-wide networks, 
community bulletin boards, and global networks) should be easily accessible and easy to use in order to 
facilitate communication between and among such groups as: students, teachers, administrators, parents, 
universities, public libraries, states and federal agencies, businesses, and community groups. Information 
technology and curriculum are intrinsically linked and ever changing. This plan is not meant to be static, 
rather it is designed as a working plan, and our goals will necessarily change as technology changes, 
curriculum evolves, and as we move into a technology based society. 

Our Campton Community Technology Program's goal is to generate and strengthen parent and 
community involvement, allowing the school community to move from isolated classrooms and families 
to one connected, active, collaborative learning community. Parents and community members participate 
in several workshops to leam how technology can be a vital part of their lives. Additionally, parents can 
check the Internet for their student's homework at HomeworkNow Our greatest educational success will 
come when information technology becomes an integral part of all areas our curriculum and an everyday 
aspect of the students' learning process. 
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Carteret County Public Schools 
HOMEPAGE 

Carteret County is a rural, fairly isolated County on the coast of North Carolina. The Public School 
System is composed of 15 schools with a population of 8,500 students. 

The Technology Plan for Carteret County School System forms the foundation and the guide for all the 
roles of technology in the District. Support for the Plan has been widespread and effective, from top-level 
administration, faculty, staff, students, and community. Without this foundation of planning and this 
platform of support the level of technology integration now evident in Carteret County Schools would 
not have occurred. 

In review, the District has met or exceeded expectations in most areas and progress continues towards 
stated goals. Because of this progress the 1998 revision of our Technology Plan focuses more on the 
seamless integration of technology into the learning environment which is our ultimate goal. 

Earlier iterations of the District's Technology Plan emphasized the acquisition of hardware and the use of 
software as a supplement and enrichment for the traditional classroom setting. The current Plan focuses 
on the evolution towards integration of technology into a student-centered, project-based learning 
environment. In the 1 998 revision telecommunication technologies, web publishing and wide area 
networking are more in evidence. 

Accomplishments over the last three years include: 

• progressive inroads into integration of technology for instruction and learning 

• major progress towards the minimum standard of equipment in all classrooms 

• campus networks that meet or exceed State standards in all schools and Central Office-- installed 
and functioning 

• all teachers participating in technology staff development with the majority having completed 
Parts I and II as defined in the technology staff development outline 

• positive beginning to a new model for technology staff development that assists in moving schools 
towards a student centered, constructivist learning environment, which meshes with the 
overarching goals of the School System 

• additions to the technology support staff— eight instructional technology coordinators, six 
technical support technicians 

• implementation of wide area network through State Telecommunication Services increased School 
System and public support for technology in order to meet the needs of students living in an 
Information Age 
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• institution of evaluation strategies that assess progress in meeting the goals and objectives of the 
Technology Plan. 



These things have been funded or made possible through: 

• passage of a local 6.2 million dollar bond referendum for technology 

• State Technology funding 

• Title VI funds 

• Technology Literacy Challenge Grant funds 

• Universal Service Rate (e-rate) discounts 

• other local funds that include additional money for media centers 

• a portion of local and State instructional funds 

• local school fund raisers 

• contributing partners and businesses 



It is anticipated that the coming years will see the same or increased progress towards the goals, 
objectives and strategies of the School System's Technology Plan. 
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Cherokee County School District 

Alabama 

Cherokee County can best be described as a rural agricultural community whose solid economic 
environment is comprised of fertile lands and an abundant water supply. Agriculture is the county's 
largest industry, involving not only row crops but also new agri-businesses such as seed production and 
ornamental plants. Weiss Lake makes tourism a vital part of the local economy. Little River Canyon 
National Preserve, historical sites, points-of-interest, and town festivals compliment the county's appeal 
as an attractive place to visit. 

The county school system offers the more than 3500 students of Cherokee County extensive learning 
opportunities, including the traditional curriculum, fine arts, vocational training, and technology. 
Furthermore, with the many athletic programs and extra curricular activities at each school, students are 
assured a well-rounded, quality education. 

Our commitment to a quality technology program has been the single "thread" that runs through all areas 
of the curriculum. Our technology program had its beginnings some seven years ago when our local 
telephone company (Peoples Telephone Company) asked if we would like to have dial-up access to the 
Internet supplied to the schools and also be responsible for managing the server. This was a true learning 
experience for both parties. They succeeded as an Internet Service Provider, and the dial-up Internet 
access opened up the world to our students and teachers. Since that first connection, technology has been 
a main focus in our curriculum. Three years ago the Cherokee County Board of Education voted to 
provide the matching funds necessary to install a LAN at each school and tie everything back to a central 
location so that information could be shared from school to school and with the Central Office. The 
design, installation, and maintenance of the LAN's and WAN would be based on the concept that we 
would have to rely on ourselves for almost everything. We were operating on a "shoe-string" budget and 
could not afford the luxury of hiring outside consultants or contractors. In a four month period of time, 
with hundreds of hours of volunteered time eight students and two teachers wired every school in 
Cherokee County. Over 1 5 miles of Category 5 wire was installed, along with the hundreds of connectors 
used to terminate the wire ends. The eight student technology team wired each equipment rack, installed 
the patch panels, hubs, switches, Novell servers, and connected all computers at each school to the newly 
established LAN. The Cherokee County WAN consists of HDSL (786 kbs) connections at six sites and 
the four remote schools are connected to the WAN via 10 Mb wireless spread spectrum links. Wireless 
connectivity was the only affordable option and has proved to be as strong a telecommunication link as 
the conventional sites, only less expensive. With the help of Trillion Digital Communications, we 
became the first school system in Alabama to have a wireless WAN connecting schools. This hybrid 
network is the backbone of technology in Cherokee County. It is unique and one of a kind that serves the 
students and teachers well. Internet access, administrative data transfers, and e-mail are the main 
functions of the network at this time. In August 1 999, distance learning becomes a reality in our rural 
school system. Jacksonville State University and the Cherokee County School System will partner a dual 
enrollment project that will allow students and teachers to take college courses via a video conferencing 
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system between JSU (some 45 miles south of Cherokee County) and Cherokee County. 

Our newest partner, Lucent Technologies, will assist us in taking our technology plan to our ultimate 
goal. It has always been our plan to eventually bring video, data, and, audio to the desktop. With a very 
limited budget we knew that this would be something that would be years down the road before we could 
obtain this goal. With Lucent Technologies newest products, MMCX and Virtual Lecture Hall, we will 
be able to meet our goal of desktop video conferencing and even surpass it. We will now be able to share 
resources from all over the county, as well as utilizing resources from neighboring systems. We will now 
be able to offer advanced courses to small groups of students at all our school sites from a central 
location. Collaboration and sharing of ideas and best practices between teachers, students, administrators 
can now be easily facilitated over our WAN and across links to distant sites. 



We are exactly where we wanted to be, only about two years ahead of schedule thanks to the help, 
support, and expertise of Lucent Technologies, Trillion Digital Communication, Peoples Telephone 
Company and a great deal of local effort and support. 
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Cherry Creek Schools 
HOMEPAGE 



The Cherry Creek School District is changing the teaching and learning environment for teachers and 
students by applying effective instructional practices, technology uses, and technological infrastructure to 
directly target and accomplish our student achievement objectives. We began the process by developing 
a comprehensive five-year district technology plan which aims best practices in technology use directly 
at enhancing student achievement. The goals of our program are designed to ensure that students and 
teachers have modem, powerful technology that will help them apply skills necessary to thrive in today's 
digital age. We expect teachers to become developers of technology integrated curriculum and students 
to become critical thinkers, problem solvers, and life-long learners. Our goals include: 

• Using technology to enhance student achievement. 

• Engaging teachers in ongoing professional development focused on student achievement through 
technology use. 

• Establishing an electronic culture within the district. 

• Evaluating technology use in the district to ensure its application to enhance student achievement. 

The primary focus of our program is to put a critical mass of technology in classrooms where students 
can use it to apply critical thinking and problem solving skills, along with basic skills to meet the District 
student achievement objectives. To accomplish this, we use a solutions approach to purchase new 
technology. Solutions consist of hardware, software, professional development, and model curriculum 
and have evidence of being effective for student achievement. Schools write and evaluate annual 
technology plans, which focus on how technology can meet the building's student achievement goals. 

Professional development is the key to ensuring that technology is integrated with the curriculum. We 
consistently spend one third of our annual technology budget on professional development. The most 
significant piece of our professional development comes from site-based Student Achievement 
Specialists (SAS). The SAS is a certified classroom teacher who understands school reform, instruction, 
curriculum, and how technology can enhance learning. The SAS is provided with a half to a full day of 
release time daily to conduct professional development in their buildings. The District and the school 
share the funding for the SAS position. To foster and guide these professional development efforts in the 
buildings, the District provides four Teaching, Learning, and Technology Specialists (TLTS) to support 
the schools. The TLTS provides support and training for the building SAS and works under the direction 
of the Director of Technology, Stephen Cowdrey. 

The advances made in instruction could not be accomplished without the infrastructure and technical 
support provided at the district level. We have installed wide area, local area, and video networks in 
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every school. Every classroom has a high-speed connection to the Internet and all students and staff have 
e'mail addresses provided by the District. An intranet is in place to store, share and quickly disseminate 
information and resources. We have district level technical support services which, at the least, provides 
for a half day of onsite technical support at elementary schools and a full day of onsite technical support 
at middle and high schools each week. 



The success of our program lies in ongoing per pupil funding. The school programs are funded through 
per pupil capital reserve money and decentralized budgets. The money for the infrastructure was funded 
through a $5 million bond election. We recently passed a budget election of which $.8 million annually is 
designated for technology. This allows us to be completely funded from ongoing sources. 



We are currently in the fourth year of our five-year technology plan and we will write a new plan in the 
fall. Our ongoing evaluation of our first technology plan demonstrates we are successful. Our SAT and 
ITBS scores have increased over the course of our technology implementation and are at an all time high 
in the District. We are also measuring ourselves against the StaR chart from the CEO Forum, the 
National Educational Technology Standards, and the Professional Development Indicators and Seven 
Dimensions of the Milken Exchange. We are making considerable progress on every one of these 
measures. We were a site visit district for the National School Boards Association (NSBA) last year, we 
were recently featured by Apple Computer in a nationwide satellite broadcast for our professional 
development program, and we will be featured in a video salute at this year's NSBA/ Institute for the 
Transfer of Technology in Education's (ITTE) Technology + Learning Conference. 
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Chicago Public Schools 

Illinois 

Chicago Public Schools 
HOMEPAGE 

The City of Chicago has a broad and diverse population of nearly three million people and is rich in 
ethnic communities and neighborhoods. The city has a history of cultural diversity which is reflected 
through its many cultural attractions, events, and community activities. There are 77 neighborhoods, over 
40 museums, more than 150 theaters, three of the world's tallest buildings, and the world's largest public 
library. Based on the 1990 Census of Population and Housing, Bureau of the Census, U.S. Department of 
Commerce, Chicago has over one million households and accounts for nearly one- fourth of the entire 
population of Illinois. 

According to the May/June 1996 Journal of Business Strategy, Chicago ranked second in overall 
business climate when compared to nine other large and mid-sized American cities. This was based on a 
composite index which included: the number of international air destinations; cost of living; diversity of 
worker skills; gross domestic product (GDP); population; number of Fortune 500-ranked global 
headquarters; and employment-unemployment rates. The Chicago area has the third largest labor pool in 
the country ("Geographic Profile of Employment and Unemployment," December 1995, Bureau of Labor 
Statistics). 

The business community represents a large population. Revenue paid to the state generally is returned 
through various programs and initiatives. Employees provide a tax base since they reside in the City. 

CPS is the largest school district in the State of Illinois, has the ability to levy taxes as a taxing body 
within Cook County, and may sell bonds. It is a part of city government and the City of Chicago's 
economic base. The City has a MA rating in terms of its own ability to sell bonds. CPS's credit rating is 
Baal, A-, and BBB+ from Moody's, S&P and Fitch, respectively. However, a critical revenue factor that 
impacts schools - 85% of CPS students receive free and reduced lunches, which means the residential 
tax base is substantially low. 

In 1994, the City of Chicago was designated as one of six urban Empowerment Zones (EZ) by the U.S. 
Department of Housing and Urban Development (HUD). Chicago's EZ areas are the near West Side, 
Pilsen/Little Village and the near South Side. Benefits include: $100 million in Social Service Block 
Grant (SSBG) funding, tax credits for businesses located in the EZ who employ residents of those 
communities; accelerated depreciation for capital equipment purchases for EZ businesses; consideration 
for waiver from federal regulation; and new tax-exempt EZ bonds. EZ designation also generates priority 
consideration for federal grants (1990 Census of Population and Housing). 



CPS comprises the third largest school system in the United States. It's nearly 600 schools, spread across 
228.5 square miles and service over 424,000 students (CPS Fact Sheet, May 1997 - 1996-97 "Student 
Enrollment"). The racial breakdown of the student population is as follows: 
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• 54.5% African American 

• 31 .3% Hispanic 

• 10.8% Caucasian 

• 3. 2% Asian 

• .2%Native American 

Over 83% of all students, citywide, are from low-income homes and receive free and reduced lunch (see 
Appendix N). Of the total number of students, the following represents those students who require 
special services: 

• 1 5% bilingual 

• 1 1 % children with special needs 

• 5% of the students are eligible for gifted programs 

According to the 1996 School Report Card, the citywide attendance rate is 89.6% as compared to the 
state average of 93.5%, with a student mobility rate of 29% compared to the state rate of 18.8%. 

The nearly 600 schools reflect a variety of formats: primary schools, transitional schools, middle schools, 
elementary schools, upper grade centers and high schools. The system services 22% of Illinois' public 
school students. Of the more than 424,000 students, according to the 1997 Fact Sheet, CPS student 
enrollment consists of: 

• 16,125 preschool 

• 35,289 kindergarten 

• 253,026 elementary 

• 95,691 secondary 

• 20,056 special education (preK-12) 

• 1,798 alternative schools 

• 2,469 tuitioned out 

CPS employs over 45,000 people to serve its students - 42,404 in schools, 1,534 in citywide services, 
and 1,180 at the Central Service Center. There are 20.5 pupils per teacher in elementary schools and 21 
pupils per teacher in high schools. The 27,190 teachers employed by CPS have a racial breakdown of: 

• 43.0% African-American 

• 45.0% Caucasian 

• 8.7% Hispanic 

• 1.8% Asian 

• .3% Native American 

Racial breakdown of principals in the 557 schools include: 

• 5 1 .3% African American 

• 37. 1% White 1 

• 1.2% Hispanic 

• .2% Asian ^ « 
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• .2% Native American 

Each school has control of local policy and discretionary funds through a Local School Council (LSC). 
The LSC consists of: 

• 6 parent representatives 

• 2 community representatives 

• 2 teachers 



• 1 principal 

• 1 student representative in each high school 

One of the problems faced by CPS in implementing technology has been the age and condition of many 
of its school buildings. There are approximately 765 buildings, with additions and annexes. Only 14% of 
these are less than 25 years old and have adequate or near adequate electrical capacities for technology. 
Another 26% are between 25 and 50 years old and can be upgraded fairly easily. The largest number of 
schools, over 40%, are between 50 and 100 years old. The worst problems are with the 9% of the 
buildings that are over 100 years old. An additional 1 1 % were built in the late 60's, but were designed as 
temporary buildings intended to be used for only 15-20 years maximum. Additionally, many schools 
have limited discretionary dollars and are not able to fund the high cost of technology, without 
assistance. 



As a result of the Illinois School Reform Law of 1988, as applied to Chicago, the Chicago Board of 
Education is now a "reform" Board with a decentralized school system where individual schools have 
autonomy to make their own decisions. In 1997, there was a change in philosophy that provided for CPS 
business and administrative functions to support the educational units and schools within the system. 
This is a major change from past practices and represented a unique feature of CPS reform versus that of 
many other large urban school districts. 



The total operating budget for the CPS Central Services Center in FY97 is currently $2.9 billion for 
centrally managed services. Approximately 4% of this budget ($132,300,991), is appropriated for 
technology: Learning Technologies' share is $3,428,956; the ISBE Hub is $622,035; Department of 
Libraries is $517,100; and Telecommunications is $12,250,000. Management Information Systems is 
$1 16,000,000. Nearly $64 million is allocated to establish the Wide Area Network (WAN). The Learning 
Technologies Department of CPS has received bids to develop and install a WAN that will tie all 
Chicago Public Schools and related support organizations to one centrally managed and maintained 
facility. 
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Drew School District 

Mississippi 

Drew School District 
HOMEPAGE 



Technology Goals: 

• To provide access to multimedia computers to all students and teachers for instruction. 

• To improve teaching and learning through technology and the ability to increase student 
achievement. 

• To improve curriculum delivery to help meet the needs for educational equity across the state. 

• To improve delivery of professional development so that teachers and library media personnel will 
have the training they need to help students learn to use technology and the Internet. 

• To improve the efficiency and productivity of administrators. 

Implementation: 

• Local area networks have been installed at each site using Netdays 

• Internet access is available in each library/media center and the parent center 

• Every classroom in grades K - 8 has multimedia computers that are networked with Internet access 

• Every classroom in grades 9-12 will have multimedia computers that are networked with Internet 
access by august of 1999 

• The District is attached to the State's Wide Area Network 

• Students at Hunter Middle School use technology to publish a school district and community 
newspaper 

• A computerized information hotline has been implemented. 

• The District has constructed a webpage 

• Every school has access to scanners, digital cameras, and desktop publishing software 

• The District has applied for the erate to reduce costs for telecommunications and connections to 
the Internet 

• Professional Development in technology is on-going 

• The District is implementing Tech Prep 

• Two teachers will be trained during the summer as Certified Novell Administrators 

• A variety of multimedia software is available in each media center. 

• The District has purchased accumulator software for electronically transferring student data over 
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the Wide Area Network 

• The District has purchased Student-Level Database software 

• Teachers are using HyperStudio and PowerPoint presentations to enhance instruction 

• Every teacher and student has access to the EBSCO library database. 

• Internet ready laptop computers are available for loan to parents. Internet access is provided 
through the District's dialup access 

• An Americorps volunteer was used to help teachers and students with technology 
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East Detroit Public Schools 

Michigan 

East Detroit Public Schools 
HOMEPAGE 

The community of East Detroit Public Schools has been extremely supportive of the integration of 
technology into the teaching and learning processes. The school district serves 6,800 students in grades 
K-12. There are eight elementary schools, two middle schools, one high school, one alternative high 
school center as well as preschool and adult education programs. The success of their educational 
technology program is due to the following factors and initiatives: 

Planning: 

• Developed a .70 person Technology Task Force with representation of every faction of the 
community. 

• Developed a Strategic Long-Range Plan and Integrated Curriculum that was based on the 
Technology Task Force identified beliefs about technology. 

• Hired a technology designer to build a state-of-the-art infrastructure for data, video and voice. 

• Analyzed the curriculum in terms of student s. monies and identified curricular and application 
software that could be integrated into units and lessons to improve student learning. 

Building the Infrastructure 

• Purchased and mounted large monitors in each classroom. They are connected to the video 
infrastructure to display video signal from the classroom computer, media retrieval system (banks 
of VCR's, laserdisc players, and CDI's), cable television, sat elite transmissions, educational 
television, distance learning programs, and school broadcasts. 

• Purchased and installed telephones with access to voice mail in every instructional and office 
location in the district. Teachers are able to call for help in case of emergency, make calls to 
parents about students progress, call for assistance with the classroom technology, and connect to 
voice mail to leave curricular updated and class information for parents to access 24-hours a day. 

• Installed a data network that provides Internet access, access to reference and circulation 
information (from school, public and university libraries) as well as data exchange to and from 
every classroom in the district. 

• hased and installed a new multimedia computer system and color inkjet printer for every 
instructional room in the district. 

• Purchased and installed new multimedia computer systems, laser and color inkjet printers for 
every computer lab in the district. 
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• Purchased over $530,000 worth of software that is highly correlated to student outcomes. 



Educational Technology Leadership Initiatives: 



• Hired a Director of Educational Technology to provide leadership for both the instructional and 
management technology programs. 

• Hired 8 lull-time media center teachers for the elementary schools to teach information technology 
to students 60% of the time and 40% of the time they consult with classroom teachers to assist 
them in effectively integrating technology into their lessons/units. 

• >Hired 4 full-time computer resource teachers for all secondary schools whose primary role is to 
work with classroom teachers to effectively integrate technology into the curriculum. 

• Hired a lull-time technical resource facilitator to manage the video and data networks, manage the 
elementary media retrieval system, facilitate the use of the district student record-keeping system, 
coordinate the repair of district computer-related equipment. 

• Hired a lull-time assessment/evaluation administrator to begin to develop effective strategies to 
evaluate whether the integration of technology into the curriculum is improving student learning. 

Staff Development 

Since 1991, the district has offered many staff development opportunities for staff members. From the 
spring of 1997 until the present time, staff development schedules have been sent out quarterly that offer 
opportunities nearly every day- Monday through Thursday, as well as some Saturdays. 



Grants Received 

Since 1996 the district has received one Goals 2000 and three Technology Challenge Literacy Grants for 
software acquisition and staff development that total more than $880,000. Technology Volunteer 
Program 

In 1991, the district began a volunteer program called C.A.S.T. which stands for Computer Aids for 
Students and Teachers. The program began with 5 volunteers and has expanded to over 150 active 
volunteers. CAST Volunteers help in every aspect of the educational technology program. The district 
recently won an award for this program from the Michigan Association of School Boards (MASB). 

-###- 
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East Detroit Public Schools 
Administration Building 
15115 Deerfield 
Eastpointe, Mi 4802 1 






District Information: 

Employment; Board of Education; Alumni; Street 
Map; District Calendar; EDS Credit Union 



Elementary Schools: 

Links to the elementary school buildings. 



Adult & Community Education: 

Links to the Community Schools Program and 
the Kellwood School. 

Register of Friends: 

Reestablish contacts with old friends and 
classmates. 




Phone Directory: 

Building and Administration telephone and FAX 
numbers. 




Secondary Schools: 

Links to the middle and high school buildings. 




Special Services: 

Information about Special Education in the district. 




News & Features: 

Events, accomplishments, organizations and news 
about the district; KISSED - Child Safety Program 
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East Detroit Public Schools Home Page 




Department of Educational Technology: 
Technology Literacy Challenge Fund Grant; 

Ed. Tech. Program; Long-Range Plan; Integrated 
Lesson Plans; 



Special Reports: 

Special EDPS Report to the Community 



Community Resources: 



| City of Eastpointe | Eastpointe Memorial Library | 

City of Warren | Warren Public Library | 

Macomb ISP I Macomb County Schools Financial and Enrollment Information 
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East Detroit Public Schools 

15115 Deerfield 

Eastpointe MI 48021 

Tel. (810) 445-4400 

Comments to: 
(webmaster@eds.misd.net) 
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Edgewood Elementary School 

Muscogee County School District 
Columbus , GA 

Edgewood Elementary School 
HOMEPAGE 

Six years ago, the leadership and faculty at Edgewood Elementary School made a commitment to 
provide our students with the teclinology that we considered imperative for success in the twenty-first 
century. Our progress in this endeavor has not come easy, nor has it been painless. Since our school 
district provides no technology budget, the funding of our goal has been, and remains, a major concern. 
Primarily, our technology has been purchased through the wise utilization of Georgia State Lottery funds 
and an initiative by the Georgia Department of Education called Pay for Performance. We feel that we 
have made prudent purchases of hardware, software, and provided our staff with sufficient training to 
feel comfortable with the technology. 

As we began to focus on providing our students and staff with state of the art technology at Edgewood, 
our hardware consisted of five Apple 2E's on roll around carts. These computers were shared by 2 1 
regular classrooms and various special programs. The students and teachers used these computers mainly 
for drill and practice and games. 

When the Georgia lottery was established, the funding for technology became a very real possibility. 
Within 2 years, we were able to pun base one Macintosh computer for each classroom, automate our 
media center, and purchase the Accelerated Reader Program. Lottery funding has also provided a satellite 
dish, two fax machines, and four Internet connections. 

Edgewood is one of two schools in the state of Georgia to be awarded Pay for Performance money each 
year since th • " orgia Department of Education incentive program was initiated five years ago. 

Targeting tecu ,.iOgy as our main objective, the professional staff at Edgewood has voted each year to 
put our money oack into the school, rather than take the rewards home. Thus, our faculty has contributed 
more than $100,00.00 toward technology in the building. Through this money, over the past five years, 
we have provided to each classroom; two Macintosh computers, one printer, one scan converter, and a 
27" color monitor. We have established The Edgewood Technology Center, consisting of 24 IMac 
computers, one Internet connected teacher station, and one data projector. We have supplied our staff and 
students with four Powerbook laptop computers, three digital cameras, and two scanners. Broadcast 
equipment, enabling students to create and edit videotaped productions was also purchased with Pay for 
Performance funds. 

At the onset of our technology initiative, training was driven only by individual teacher motivation. A 
few computer courses were offered through staff development, however, there was no specific focus. 
Four years ago, the Muscogee County School District hired five technology specialists an established a 
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designated site for technology instruction. The following year, our technology specialist offered to 
provide our faculty with "site based" staff development courses because we felt that we were ready to 
focus on Edgewood's specific goals. This preliminary training proved invaluable because, in January of 
1998, the State Department of Education implemented the InTech (Integrating Technology into the 
Curriculum) program, and Edgewood had a team of five teachers who felt ready to undergo the intensive, 
50-hour training. 



The five members of the original InTech team, in turn, trained the remainder of the faculty using the 
"Peer Teaching" concept. The instruction focused on using technology to supplement and enhance the 
curriculum already in place, rather that learning a particular piece of software. Our entire faculty 
completed the training in January 1999. Since the completion of InTech training, we feel that we are 
ready to begin to fully implement the technology we have purchased for our building. 

Evidence of the use of technology is apparent throughout the school; in the required technology projects 
for grades one through six, in the required Invention Convention projects for third grade, required Media 
Festival projects for fourth grade, required Science Fair projects for fifth grade, and required Social 
Studies projects for sixth grade. 

-###- 
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Edgewood Elementary 

I 

Columbus, Georgia 




Home of the Eagles 



j Mission Statement | 

1 • 1 

i The mission of Edgewood School is to boldly prepare students for the future j 
l toy providing an atmosphere of excellence in which children and adults are 
j nurtured as individuals and challenged as risk-takers in the learning process. 1 

i 



■ Background 

j Edgewood is located in Muscogee County, Georgia approximately 100 miles SW of Atlanta on the 
! eastern bank of the Chattahoochee River. The school serves 480 students in grades K - 6. There are 20 . 

regular classrooms, a self-contained learning disability class, a gifted and talented resource teacher, an 
SIA teacher and a Reading Recovery teacher. In addition to a full-time principal, media specialist and | 

! school guidance counselof , we have a part-time assistant principal and part-time music, art, PE, j 

i orchestra, band, and remedial education teachers. Edgewood also serves as the Mini-Magnet for the 
I visually impaired for our school district and staffs one full-time V.I. teacher and two paraprofessionals. J 



j 
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Edgewood Elementary 

j ~ 7 Technology j 

i Edgewood is a Macintosh school. The Techno logy Team, which is a component of our site based 
management, addresses the technology issues that face our school. There are three Macintosh computers, | 
one of which is connected to a wall mounted monitor for large group display, and an Imagewriter printer j 
in each classroom. In addition, the majority of classrooms also have an inkjet printer. The computers i 

range from LC575s to iMacs. While the Media Center is automated with 3 look-up stations, there is no ! 

school-wide network. A, variety of equipment is available including flatbed scanners, digital cameras, 
video cameras an A video editing equipment Four PowerBook 5300s are available for teacher use. j 

A student production lab has been established within the last year. We have 24 iMac computers, 
scanners, inkjet printers, and a teacher station connected to a projection device for large group display, 
i Classes are scheduled on a regular basis with additional time slots available on an as needed basis. 

i Each classroom’s Wall mounted monitor (including a VCR) is connected to the distribution system and in 
J conjunction with 6ur satellite dish, allows for video use throughout the building. Additional video 
equipment is available including three camcorders and editing equipment to enhance video capabilities. 

A Broadcast Club has been organized as an extracurricular activity to provide opportunities for students 
to become proficient in the use of video and editing equipment. 

Internet access is through Mindspring, Inc., an Atlanta internet service jirovider and is available in the 

! Media Center and'computer lab. 

\ \ 



Cool Projects At Edgewood 



Construction 



We would like to hear from teachers and students. 

Email us now! 
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Evergreen School District 

Washington 

Evergreen School District 
HOMEPAGE 



BACKGROUND 

Evergreen is a suburban school district with approximately 21,000 students housed in twenty-seven 
schools. A rapidly growing district, Evergreen will open its third high school in the fall of 1999, and a 

middle school and an elementary school are under construction. 

\ 

EVERGREEN’S TECHNOLOGY IN THE 1980s 

The district has long supported an innovative, technology-infused vocational/technical program and in 
1983 had the first high school level word processing lab west of the Mississippi. Student and fiscal 
management systems are linked to the State of Washington system, WSIPC (Washington State 
Information Processing Cooperative). In the late 1980s Evergreen School District begin installing e-mail 
and voice mail systems and creating local area networks to support these services. In 1989 a district 
junior high school was awarded a $500,000 21st Century School grant, marking the first time that 
technology was deployed to every classroom within a school. 

TECHNOLOGY PROGRESS IN THE 1990s 

With the formation of the District Technology Committee in 1990, Evergreen SD began a thorough 
assessment of the role of technology to increase student learning and staff productivity. Following the 
lead of this committee, Evergreen began its steady progress in the 1990s to deploy technology to every 
classroom and office in the district. 

FUNDING 

A Capital Needs Bond approved by the voters in 1992 allocated $7,865,000 for technology, allowing the 
district to provide a base level of technology and connectivity to every school. A second bond approved 
in 1994 provided an additional $14,704,250 for technology. In February 1999 a capital technology levy 
for $6,840,000 won 57% support of the voters but failed because it was short of the 60% super majority 
required by the State of Washington. 

SCHOOL LEVEL PLANNING FOR TECHNOLOGY 

To access the technology funds approved by the electorate, each school developed a comprehensive, long 
term school improvement and technology plan. School technology plans included educational goals, 
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needs assessments, staff development, software selection, and evaluation of educational effectiveness. 

DISTRICT LEVEL PLANNING FOR TECHNOLOGY 

To meet our sustainable vision of technology used to support effective teaching and business practices, 
Evergreen School District approaches technology from a return on investment (ROI) viewpoint. When 
considering the deployment of technology for a particular program or task, managers calculate the total 
cost of the implementation. For instance; what is the actual cost when considering equipment and 
software purchase, educational benefits, required facility modifications, staff training, technical support, 
ongoing maintenance, and eventual replacement. When it can be determined with confidence that the 
proposed technology can perform the tasks better, or faster, or cheaper, or to higher levels than with the 
traditional methods it replaces, then Evergreen will proceed with its implementation. 



EVERGREEN SCHOOL DISTRICT TODAY 



Approximately 4,500 computers are used in the district and all are networked and have Internet access. 
Each classroom has a high speed Internet access at three or more locations and a phone with district 
voice mail and direct dial functions. A systematic plan was followed to provide adaptive equipment so all 
students, including those with handicapping conditions, have equitable access to current technology. A 
computer services department provides helpdesk assistance and technical support in-house. Technology 
staff development and curriculum integration is a focus of Evergreen School District and is provided by 
an instructional technology department. 
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Fabens School District 

Texas 



Fabens School District 
HOMEPAGE 



eeZone TEXAS is an online community in which teachers collaborate with research organizations and 
the private sector to design and evaluate new best practices for using technology effectively in the 
classroom. Our participants, currently numbering nearly 50,000, represent 22 school districts, five 
regional education service centers, three universities, four research centers, and one private sector 
partner. Our mission is to serve as a cooperative of researchers, practitioners, and students, who together 
will help define, publish, and promote replicable strategies for improving education with effective 
technology practices in the nation's classrooms. 

eeZone TEXAS uses a Web-based software application to deliver interactive content and technologies ~ 
that help students develop critical thinking and analysis skills. Our cooperative uses the Internet to 
coordinate activities among participants, who are widely distributed across the state. The application runs 
on a central server, and participants access the content and interactive tools using networked computers. 
The server-side application manages user authentication, access, and data tracking, and the client 
applications run in a Web browser. The Texas Education Agency is evaluating this distributed 
client-server model for delivering electronic instructional materials as a more flexible and cost-effective 
alternative to traditional textbook publication and distribution. 

Students collaborate in teams within their school and across the state as they work through interactive, 
inquiry-based projects, which are accessed online. The projects use environmental education as an 
integrating curricular context, as it naturally lends itself to interdisciplinary team teaching that supports 
the core subject areas. Existing and forthcoming projects include A Virtual Tour of the Edwards Aquifer; 
a Virtual Wildflower Collection; Using GIS Technology to Explore Earth Systems; A Sustainable 
System for Industrial Water Re-use; Designing Air Quality Improvement Strategies for Texas; An 
Investigation of Campus Littering Behavior; A Simulated Landfill Siting; and Perspectives on Graffiti. 

Teachers join interdisciplinary teaching teams as part of the program's professional development 
activities. Curriculum and instruction specialists, assessment specialists, and teachers form working 
groups to propose new interactive project ideas and revise existing materials. The eeZone TEXAS Web 
site provides group support software to facilitate these online collaborations. As they work through the 
curriculum design process, teachers develop new skills, inform their teaching practices with new 
education research, and emerge as strong school leaders. 

Scientists and researchers at the state's universities and agencies contribute expertise and the products of 
their research work, whether that be in the field of environmental science or in education. eeZone 
TEXAS provides a forum for these experts to share their extensive research and development, which has 
been funded through previous public expenditures. Working with the curriculum design teams, these 
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experts assist in creating new inquiry-based educational projects. Modifying research products to be used 
in 6-12 grade classrooms can offer useful feedback from the field; it also encourages researchers to 
consider outreach and dissemination possibilities when designing their research projects. 



Evaluators gather much of their data through online instruments that examine how the technologies are 
being integrated into the school curriculum. These online instruments provide rapid feedback so 
participants may refine their teaching strategies throughout the year. 



In the coming year we intend to expand the eeZone program to additional states, expanding access to our 
interactive content and online professional development resources. 

UMM 

-###- 



[Evergreen School Districfl 




[Gold Oak Union School District] 



4G 



IP 



://www.ed.gov/Technology/TechConf/1999/profiles/fabens 



.html (2 of 2) [4/24/2001 2:31:31 PM] 



O 






Gold Oak Union School District, CA 

The Secretary's Conference on Educational Technology-1999 



Gold Oak Union School District 

California 

The Gold Oak School District is comprised of two school. Gold Oak serves students in grads K-5 and 
Pleasant Valley the 6-8 population. At present, Pleasant Valley School has a PC computer lab with 29 
Pentium computers to the Internet and a Mac Lab with 1 6, 5200 Power Mac's also connected to the 
Internet. Teachers at Pleasant Valley have access to both labs on a regular basis. All teachers at Pleasant 
Valley have at least one network capable computer in their classroom except for the P.E. and music 
teachers. At Gold Oak the computer lab has 28 computer stations (a combination of Pentium PCs and 
Mac LC 580's) with Internet connection, which also can be accessed on a regular basis by all teachers. 
More than 60% of the teachers at Gold Oak School have at least one network capable computer in their 
room. 

Goals 

• Use technology as a tool for teaching and learning 

• Research the effectiveness of technology as it relates to standards and curriculum 

• Empower students through the creation of a student level technology support system 

• Empower parents by using technology to increase literacy skills and give access to student 
information and Internet resources 

• Make technology available in all areas of the school community 

WHAT HAVE WE LEARNED? 

• The classroom environment must be "technology-rich" in order for change to occur. 

• Innovation is driven by the sharing of great ideas. 

• Dialogue, whether on-line or face-to-face, becomes an avenue for discussing both successes and 
frustrations. 

• Training tied to standards and to classroom instruction is makes learning meaningful. 

• Technical support is essential to student/teacher use of technology. 

• Technology in the classroom facilitates constructivist practices. 

THE INSTRUCTIONAL TECHNOLOGY EXPERT PROGRAM (ITE) 

• Ninety teachers from across the district are trained in the ITE program. 

• Thirty participants form each of three consecutive cohorts who receive 9 full days of training as 
well as after-school workshops throughout the year. 

• Full day training for ITE teachers focuses on technology integration with curriculum and district 
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standards in the area of language arts. Workshops focus on the use of specific technology tools. 

* • ITE classrooms are equipped with a minimum of six networked multimedia computers, color 
inkjet printer, laser printer, digital camera and flatbed scanner. 



THE TECHNOLOGY ASSISTANCE PROGRAM (TAP) 

• The TAP program trains thirty students per year at the 4th and 6th grade levels. 

• These students are selected by ITE teachers from each cohort and serve as a student level technical 
support system. 

• Trained on a bi-weekly basis, they take classes before or after school which focus on classroom 
software as well as basic computer and network troubleshooting. In the near future, these students 
will have the opportunity to become Apple Certified technicians. 



THE PARENT PROGRAM 

• The parent program offers classes to parents on an eight-week course of study. 

• Classes are in Spanish and English, and held after school hours and on Saturdays. 

• Topics in the parent program range from computer basics and classroom software, to e-mail, the 
Internet, and multimedia. 

• The purpose of the parent program is not simply to offer classes in skill development but to 
encourage parents to serve as a support mechanism for technology-rich classrooms across the 
district. 



THE PROFESSIONAL DEVELOPMENT CENTER (PDC) 

• The PDC is a state-of-the-art facility for staff development in technology. All ITE training, the 
TAP program and parent classes are held in this center. 

• The PDC is comprised of thirty networked, multimedia computer workstations, scanners, digital 
cameras, video cameras, and laser printers. 

• The design of the center fosters collaborative learning, as each group of four workstations are 
interconnected. All workstations are ergonomically designed such that monitors reside below the 
level of the desk. This allows for easy communication between group members. 

LAPTOP LOAN PROGRAM 

To encourage learning at all moments, laptops are made available to teachers, parents and students 
involved in the programs mentioned above. Teachers use the computers for instruction and network 
management in the classroom, and to access the network from home. Students and parents use the 
computers to reinforce what is learned in class and to communicate with teachers via e-mail. 

COMMUNITY CONNECTIONS 
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In the coming year, community centers across Lennox will be provided with a computer kiosk for access 
to the Lennox Network and the Internet. The first to receive such access is the Healthy Start Community 
Center. Future locations include the Lennox Public Library, the Lennox Sheriffs Station, and the 
Richstone Family Center. 

it i Id j 
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Helena School District 

Montana 

Helena, Montana, nestled at the base of the Rocky Mountains, is Montana's capital city. The Helena 
School District, which serves 8188 students, includes two high schools, two middle schools, ten 
elementary schools, an alternative high school, a transitional middle school and an Instructional 
Materials/Technology Training Center. The district employs an exceptionally well-educated staff of 500. 
These highly motivated professionals serve our children, schools and community with incredible 
dedication and compassion. Helena Public Schools has a long tradition of excellence in all areas whether 
it be academics, the arts, technology, or sports. We continue to strive to meet the mission of the Helena 
Public Schools, which is: to challenge our students to maximize individual potential and to become a 
competent, productive, responsible, caring citizen. 

District Technology Plan for the School District was begun in 1990 and has been completed. It is a 
dynamic document that is constantly revisited and revised. Each school includes a technology section in 
their Annual School Improvement Plan. Each building's Annual School Improvement plan must address 
the role of technology in enhancing teaching and learning. Further, the plan must describe how 
technology will be acquired and used to maximize improved student learning. 

The District established a District Technology Coordinator position in January of 1997. This position 
oversees and coordinates the implementation of technology in the District. A network manager position 
was created and filled during the 1998-99 school year. This position is responsible for security at all 
levels and for all systems. Network authentication protocols are being established and implemented. 
Business applications, student management applications and library applications are password protected. 
An Elementary Technology Specialist position was established in 1994 and in 1997 responsibilities of 
the Training Center Coordinator were included in that position. Library Media Specialists in some 
locations have accepted responsibility for building level technology support. Each high school has a 
technology teacher position and the two middle schools have also established technology teacher 
positions. 

The District Technology Training Center was established in the fall of 1996. An extensive Professional 
Development program was developed and professional development opportunities for District teachers as 
well as teachers from surrounding Districts and community members continues today. The Center 
operates in the evenings, on weekends and throughout the summer months. Non district staff are charged 
a nominal fee to help defray overhead costs. 

The District has invested in and implemented local area networks in its high schools, middle schools and 
the first five elementary schools. The remaining elementary schools are scheduled to be wired during the 
99-00 school year. Sites with a local area network are connected in a wide area network using wireless 
communications tools. The District selected and implemented a District wide K-12 Student Management 
System during the winter of 1997-98. 
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High school and middle school libraries are automated and provide a wide array of resources both 
electronic and print which support teaching and learning. Elementary libraries will be fully automated in 



the fall of 1999. 



All students and staff in our high schools have comprehensive access to the tools of technology. Access 
includes classroom, mini-lab, full lab and library lab access. Further all high school instructional sites 
have both local and wide area network access. The two middle schools and ten elementary schools have 
centralized access to network resources and varying levels of access to technology tools. 

Each school has a District web site which highlights school activities and student work. Each high school 
web site also includes instructional information provided by classroom teachers. Email accounts are 
provided for all staff members. 

A set of essential technology skills has been identified at each grade level. Work continues to insure that 
technology is used as a tool to support teaching and learning and that fundamental skills be used in the 
context of the curriculum. Efforts at evaluation have tended to be tied to specific programs. The 
Accelerated Reader Program has been extensively evaluated over a period of five years. Student 
achievement has been documented through a series of standardized tests, student performance records 
and other assessments. Students participating in the Problem Based Learning project have provided 
anecdotal evidence of improved achievement through oral assessments. Additionally, students 
participated in pre and post assessments which measured their technical abilities. No evaluation has 
established an absolute and direct link between student achievement and technology integration. 
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Henrico County Public Schools 
HOMEPAGE 



Technology 

The Department of Technology is responsible for the technical implementation, maintenance and support 
of instructional technology initiatives, ongoing technology programs and for technology production 
services in the school division, kindergarten through grade 12. 

Department of Technology functions include: 

• Instructional and administrative computing network design, installation, repair and support 

HIGHLIGHTS: 

Henrico County Public Schools has historically been and continues to be a state and national leader in the 
application of technology to K-12 instructional applications. Henrico began televising high school 
distance learning courses in 1984 and has implemented computer Integrated Learning Systems (ILS), 
videodisc applications for Science and Mathematics, a student/teacher electronic mail system and is 
currently working on the implementation of a county-wide Metropolitan Area data communications 
Network (MAN) utilizing an Instructional Cable Television Network or I-NET. This system will provide 
data communications between all schools and direct internet access to all computers on school networks 
while avoiding typical high costs for telecommunications charges. 

Major Technology Initiatives already in Place in Henrico Schools 

• Elementary Classroom Computer Initiative 

• Institutional Cable Television Network (I-NET) and television distribution systems in each school 

• Two-way video and audio instruction between school sites for specialty/low enrollment courses 

• Satellite reception capability in each school 

• Library Automation Initiative Implementation (all. schools) 

• Secondary Technology Initiative 
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Chariho County Schools 
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1. Implementation of Technology 

We are a K-4 elementary public school. Hope Valley is one of four elementary schools in the Chariho 
Regional School District. Please see Hone Valiev Elementary School Web Page for profile, belief 
statements, and objectives. We implement technology in every grade level, in every subject area. Our 
work reflects broad-based use of technology from electronic information gathering emphasizing 
information literacy skills, all the way through to the formal product/project presentation. Through 
inquiry based research, students are developing more advanced questioning techniques and asking 
essential questions that lead to every more research and inquiry and a deeper understanding of the subject 
matter. 

The Chariho District Web Site provides access to curriculum resources, school web pages, classroom 
pages, and student work. It has evolved over the past three years of its existence and reflects the district's 
increasing implementation of technology. Quality of student work using technology is evident through 
their contributions toward developing award-winning websites: 

• Sands of the World : awarded TechCorps, WNET, and recognized in Classroom Connect 
Magazine, March, 1999 

• "Internet Sites for Kids"is indexed by several other school web pages in the state as a summary 
resource site for child-oriented search engines. Twelve outside websites throughout the country 
index Hope Valley School's web site. EduChoice Award for providing outstanding educational 
material on the web. Tech Corps (a cooperative association of commercial and educational 
interests) will showcase this site at its Web Nite 2.0 in June. 

2. Wide Area Networking and Local Area Networking: 

Rhode Island's RINET provides connectivity to all teaching professionals in the state, with the cost of 
service assessed to the various school districts. Through RINET, teachers have internet and email 
services. Locally, the Chariho Regional School district operates its own server, and affords connectivity 
to all schools in the district. Every professional and para-professional has an email account. Buildings are 
networked, and every elementary classroom has a pentium computer connected to the network. Teachers 
who participated in the RITTI program have laptop computers, which also can access this network. 
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3. Preliminary Evaluative Work of Hope Valley Elementary School: 

• Information Works School and District Achievement and Performance Reports data based on 
standardized testing indicates that students at Hope Valley performed better than expected given 
demographic indicators. 

• 1 998 SALT Report for Hope Valley Elementary School. 

• 1999 SALT Visit Report(not yet published) cited exemplary integration of technology into the 
curriculum at Hope Valley School. In this SALT report, the Hope Valley web site was described 
as an exemplary source of information for parents, teachers, and community 

• Teacher observation has found that technology has motivated and excited students, even those 
who previously disliked school work or were indifferent. Students have been forced to employ 
higher order thinking skills in evaluating and utilizing the information they encounter. 

4. Timeline for Technology Implementation: 

• District Technology Plan 1993 

• District Technology Initiative Phase I, June 30, 1996 

• District Technology Initiative Phase II, June 30, 1997 

• Champlin Grant June 1998 

• District Technology Initiative Phase III, June 30, 1998 

• District Technology Curriculum 1999 is in the final stages of approval. It reflects current practice 
as well as prescribes additional and continual technology integration. Will serve as basis for 
evaluation of programs beginning September 1999. 



5. Professional Development: 



Staff needs assessment area of technology training has been done annually for the past three years. 
Workshops conducted by district staff, mostly RITTI trained, and financed though RITTI grants and 
district funds, have addressed every topic in which teachers felt they needed training. Impetus for 
wanting more training comes from seeds planted by RITTI, and it drives that training to a point where 
teachers increasingly progress as they realize the potential impact of technology in education. 
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Hundred High School 



West Virginia 



Hundred High School is located in a small rural community in West Virginia with an enrollment of 
approximately 170 students enrolled in grades 9-12. 

During the 1998-99 term, the school began with the implementation of the NetSchools solution. With 
this program, every student in the school was provided a durable StudyPro™ laptop, which could be used 
both as a stand-alone computer and as a wireless, networked computer. The laptops were loaded with a 
number of applications including Microsoft Works word processing, spreadsheet and database software 
as well as a popular browser, email, a presentation creation package and a graphing program. 

Not only does every student have a laptop, but also every teacher was given a laptop for instructional as 
well as classroom management tasks. In addition to the same application packages, teachers were 
provided with an email package, an Internet browser, and special classroom management tools. These 
tools include AIS Desktop Monitor which allows each teacher to control student laptops from their desks, 
and the Curriculum Browser/Search Tool developed by NetSchools for access to a large database of 
tested Internet sites that have been correlated to the West Virginia Instructional Goals and Objectives. 



The teachers have incorporated this technology into daily classroom instruction in a number of ways. 
Through the use of several application software packages students are able to complete assignments 
using the application tools, using email for receiving and submitting assignments or collaborating with 
other students on group tasks, accessing information using the Internet browser, creating presentations to 
share with others and participating in mentoring projects. 

Through the Wetzel County Board of Education, an important vehicle for enhancing school home 
communications has been added. Students and teachers have local dial-up access to the school server 
which provides the opportunity for accessing files stored on the server as well as the Internet from home 
during the evenings and weekends. Parents/guardians were included in the initial training and 
dissemination of the technology, and were encouraged to communicate with teachers and students using 
the technology provided to their children. Parents also have the opportunity to observe what their 
children are doing in the classroom by viewing assignments and files on the laptops. 



To increase the level of success for this program, the package purchased from NetSchools provides 
on-site teacher and student instructional training, and technical support throughout the first year of 
implementation. Evaluation data of the program is being collected on a continual basis to establish the 
effectiveness of the program in terms of student learning as well as teacher progress in delivering 
instruction through this new technology model. 
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Jennings School District 

Missouri 



Jennings School District 
HOMEPAGE 

Beginning in the summer of 1997, the Fairview Elementary School of the Jennings School District was 
selected to participate in a Missouri pilot project known as MINTs, Multimedia Interactive Networked 
Technologies. MINTs trained the teachers, delivers high-speed Internet connectivity into the classroom, 
and places technologies on the teachers' and students' desks. The purposes of the MINTs project were to 
eliminate the technology barriers, change (reform) teaching styles and strategies, and significantly 
improve student performance in historically high "at risk" and low achieving urban school districts. 

Missouri funded two classrooms in each of six high "at risk" urban school districts. Two classrooms— a 
fourth grade and a fifth grade— in Fairview participated in the technology project. Each class was 
provided with a student to computer ratio of two to one, a teacher workstation consisting of a computer, 
electronic white board (smart board) and projector, two printers (one color and one black and white), a 
scanner, and a video camera. In addition, each classroom was connected to the Internet with a 1 O-Mbps 
line. 

One classroom focused on student improvement in science and the other classroom focused on the 
improvement of reading and writing skills. However, r as will be evidenced in Section 3, there was 
significant improvement in student performance in other areas as well as significant reform in teaching 
styles and strategies. 

In addition to the technology available in each classroom, new student desks (workstations) were 
designed and purchased. The student desks were ergonomically designed and also provided workspace 
for a student on each side of the monitor. The newly designed student desks are visible in the video 
accompanying Section 4. 

The project is now being expanded to 14 additional (16 total) classrooms throughout the district. 

Fourteen teachers have volunteered for the ongoing teacher training program necessary to replicate the 
hi-technology classrooms. Each of the 14 classrooms will be equipped essentially the same as the pilot 
classrooms with the exception of the connectivity. The new classrooms will have a T-l connection to the 
Internet. 

At the same time, the district was piloting the technology in the classrooms and expanding the hi-tech 
classrooms throughout the district, the district was also expanding its technology support staff. Therefore, 
for the 1999-2000 school year, the district will have 16 hi-tech classrooms grades 4-12; two technology 
instructional specialists will be available full-time to insure efficient integration of technology into the 
curriculum; and three technology support staff will be available to insure efficient operation of the 
classroom technology. 
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NOTE: The district has two to three modem computer labs in each building. The labs are not considered 
innovative and are, therefore, not part of this profile. 
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Arizona 



Kayenta is a small rural Navajo community located in an isolated region in the northeastern comer of 
Arizona, near the magnificent Monument Valley. This school district serves 2600 students from Kayenta 
and several other smaller, more rural communities. The nearest public library is 100 miles away, while 
the nearest museums, bookstores, and universities are 150 miles. This isolation has provided the 
motivation to use technology to assist in increasing literacy, while permitting students to sustain critical 
elements of the rich traditional life of generations of Dine. 



The Kayenta Unified School District (KUSD) committed itself to the implementation of educational 
technology about ten years ago under the leadership of former superintendent Bob Roundtree. At that 
time, a long-range technology plan was outlined that specified infrastructure development. This 
aggressive technology plan began with district administrators, expanded to office staffs, and eventually 
extended to every classroom. Thus the long-range plans for infrastructure were put into place, along with 
technology support systems. 



Presently, all six school and administrative buildings, and all classrooms, offices, and administrators are 
connected to an Intemet/intranet email system. There are currently 550+ desktop workstations; both 
Macintosh and Windows based. Infrastructure upgrades will continue within buildings and classrooms 
during the coming school year. 



Kayenta's Staff Development Office began training teachers in the personal use of computers, Internet, 
and Intranet email three years ago. During the 1998-99 school year teacher's skills were assessed using a 
locally developed assessment instrument based on a five-stage model of Classroom Technology Use. 
Initial results show that almost all teachers are comfortable with some use of computers, but that few 
have fully integrated technology into classroom instruction. Each teacher's classroom technology use is 
now focused through an Individual Technology Action Plan that will be completed and renewed 
annually. 



An initial internal evaluation indicated that in order to maximize student learning the efforts made 
towards training teachers would need to be expanded to include students, support staff and eventually 
community members. This presents an uncommon challenge, since there are no other sources of 
multimedia technology available to the larger community population. 



Determining ways to complement traditional instruction and values, while using technology to increase 
literacy and reduce isolation is the compelling challenge faced by Kayenta Unified School District and 
other rural isolated school districts. 
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Kentucky 

Kenton County Public Schools 
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With the Kentucky legislative action of 1992 awarding schools district Ed-Tech funds to be matched 
dollar for dollar by the local school district, Kenton County Schools began a technology implementation 
to support the curriculum. 12 months were spent in organizing a district technology committee and 
planning for the future implementation. It was decided that as funds were disseminated from the state on 
an annual basis, we would purchase a few computers and printers and combine the remaining 1 st year 
funds with the 2 nd year funds and cable all of the buildings. In combination with construction going on in 
the district, we felt that creating the infrastructure was the key to a long-range success. All future funds 
would be assigned to the schools based on their enrollment and inclusion of technology in the building 
consolidated plan. 

After completing the cabling to each classroom, library and administrative area, Kenton County Schools 
began adding school buildings to the WAN (wide area network). Professional development began by 
focusing on writing and using a word processor as a tool for that activity. Teachers were also trained in 
the instructional uses of E-mail. As teachers acquired these skills PD began to focus on instructional 
aspects of the Internet. As each year passed a new concept, either tool or supplemental content software, 
was introduced into the professional development plan. 

As more equipment was installed in each school the district hired several full time technicians. While 
that took care of hardware failure, it did not address the day to day operator issues that occurred. The 
district staff implemented Technology Teams in each building to build capacity in each school. We 
complemented that with students who became part of the Student Technology Leadership Program. We 
found it necessary to have a team of teachers, staff and students that were on site and could support 
minor failures and one-on-one training needs. In organizing this support structure, we held annual 
summer training sessions for each group, correlating the training's to cover the same material. 

With the addition of the TLCF (Technology Literacy Challenge Fund) offers, Kenton County Schools 
was able to purchase items other than computers and printers that added to the curriculum. The first year 
digital cameras and scanners were purchased. The professional development was focused on using that 
equipment to create web pages. It exceeded our expectations. The second offer of TLCF was used to 
train 1/6 of the teaching staff on the instructional use of a presentation tool (HyperStudio). Each 
participant had to complete 3 projects including a student project from their classroom. A web page was 
created with links to the projects for teacher sharing. The Year 3 offer will be used to purchase projection 
devices for students to present multimedia presentations incorporating the items from Years 1 and 2. 

While each year we continue to add more up-to-date workstations and printer to each school based on 
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their enrollment, we have found that the TLCF offers have made a huge impact. As this is a district with 
ldw free and reduced lunch students (rated at 52%), we qualified only for communication reimbursement 
from the E-rate. This has reduced the annual cost of leased lines for Internet access to our schools. 



With this school year we added a teacher as a district trainer to the technology staff. This person 
conducts a weekly 3-hour technology PD based on the various tools and curriculum software available to 
teachers. She is also made available to schools, as requested, to work one-on-one with teachers during 
their planning or school created time. We currently focus on the Teacher Standards adopted by the 
Kentucky Professional Standards Board and make sure each person is informed which standard each 
training addresses. 



We are looking for measurable progress in the area of technology with our students. We participated in a 
survey from the Milken Family Foundation focusing on teachers, principals and the district's perception 
of technology and it's integration. We also survey schools annually for needs, implementation and 
perceptions. 
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Kuna School District 

Idaho 

The Kuna School District is a rural district 20 miles southwest of Boise. Although Kuna is primarily a 
farm and agriculture community, many who work in high-tech industry in Boise and surrounding towns, 
for companies such as Hewlett-Packard, Micron Electronics, and Micron Technology, have opted to live 
in Kuna, and send their kids -to the Kuna School District. 

Kuna School District's commitment to educational technology began in 1992, when district funds were 
committed to begin installing local area networks in its schools. Over the next 6 years, all schools, and all 
classrooms in the district were wired for Internet access through the building LAN. 

Integration into classroom teaching started slowly, and has exploded in the past two years. In 1994, the 
district started a pilot project, where 2-3 teachers per building were given a bank of 5 computers per 
room, and were asked to experiment with using technology in their teaching. Today, all elementary 
classrooms in the district have 3-6 computers, and all secondary classrooms have at least one computer, 
with access to more in shared labs. 

The district has a structured professional development plan for technology integration, which allows 
teachers to choose from a wide variety of training and development opportunities, including both 
technology skills and classroom integration strategies. Teachers can choose from after-school technology 
classes, student-teacher mentorships, help from teacher integration specialists available in each building, 
and informal help from grade level or department technology coordinators. 

The district is committed to continual needs assessment and evaluation of the technology program, 
including evaluating the impact of technology use on students and teachers. Starting this year, a specific 
plan is in place to yearly evaluate the impact of technology on student learning. At this point, evaluation 
efforts are rudimentary, but with more training and practice, the district expects to have valuable data to 
share with other districts on the benefits of technology integration. 

The goal of the Kuna technology program is the same as that of the district - that all students will achieve 
their highest potential. We believe that educational technology, appropriately used, has the potential to 
motivate and challenge students to not only achieve their highest potential while in our care, but to 
become competent, life-long learners. 
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Lennox School District 

California 

Lennox School District 
HOMEPAGE 



GOALS 

• Use technology as a tool for teaching and learning 

• Research the effectiveness of technology as it relates to standards and curriculum 

• Empower students through the creation of a student level technology support system 

• Empower parents by using technology to increase literacy skills and give access to student 
information and Internet resources 

• Make technology available in all areas of the school community 



WHAT HAVE WE LEARNED? 

• The classroom environment must be "technology-rich" in order for change to occur. 

• Innovation is driven by the sharing of great ideas. 

• Dialogue, whether on-line or face-to-face, becomes an avenue for discussing both successes and 
frustrations. 

• Training tied to standards and to classroom instmction is makes learning meaningful. 

• Technical support is essential to student/teacher use of technology. 

• Technology in the classroom facilitates constmctivist practices. 



THE INSTRUCTIONAL TECHNOLOGY EXPERT PROGRAM (ITE ) 

• Ninety teachers from across the district are trained in the ITE program. 

• Thirty participants form each of three consecutive cohorts who receive 9 full days of training as 
well as after-school workshops throughout the year. 

• Full day training for ITE teachers focuses on technology integration with curriculum and district 
standards in the area of language arts. Workshops focus on the use of specific technology tools. 

• ITE classrooms are equipped with a minimum of six networked multimedia computers, color 
inkjet printer, laser printer, digital camera and flatbed scanner. 
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THE TECHNOLOGY ASSISTANCE PROGRAM (TAP) 

• The TAP program trains thirty students per year at the 4th and 6th grade levels. 

• These students are selected by ITE teachers from each cohort and serve as a student level technical 
support system. 

• Trained on a bi-weekly basis, they take classes before or after school which focus on classroom 
software as well as basic computer and network troubleshooting. In the near future, these students 
will have the opportunity to become Apple Certified technicians. 




THE PARENT PROGRAM 

• The parent program offers classes to parents on an eight-week course of study. 

• Classes are in Spanish and English, and held after school hours and on Saturdays. 

• Topics in the parent program range from computer basics and classroom software, to e-mail, the 
Internet, and multimedia. The purpose of the parent program is not simply to offer classes in skill 
development but to encourage parents to serve as a support mechanism for technology-rich 
classrooms across the district. 



THE PROFESSIONAL DEVELOPMENT CENTER (PDC) 

• The PDC is a state-of-the-art facility for staff development in technology. All ITE training, the 
TAP program and parent classes are held in this center. 

• The PDC is comprised of thirty networked, multimedia computer workstations, scanners, digital 
cameras, video cameras, and laser printers. 

• The design of the center fosters collaborative learning, as each group of four workstations are 
interconnected. All workstations are ergonomically designed such that monitors reside below the 
level of the desk. This allows for easy communication between group members. 



LAPTOP LOAN PROGRAM 

To encourage learning at all moments, laptops are made available to teachers, parents and students 
involved in the programs mentioned above. Teachers use the computers for instruction and network 
management in the classroom, and to access the network from home. Students and parents use the 
computers to reinforce what is learned in class and to communicate with teachers via e-mail. 

COMMUNITY CONNECTIONS 

In the coming year, community centers across Lennox will be provided with a computer kiosk for access 
to the Lennox Network and the Internet. The first to receive such access is the Healthy Start Community 
Center. Future locations include the Lennox Public Library, the Lennox Sheriffs Station, and the 
Richstone Family Center. 
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Little Falls Community Schools is located in Morrison county in rural Minnesota. Morrison county has 
an unemployment rate of 7% which is well above state and national averages. Per capita income in 
Morrison county is $16,563 which places Morrison county 7th lowest out of 87 Minnesota counties. 

Over 55% of our elementary students qualify for free and reduced lunch. Educational attainment levels 
are low; 3 1% of Morrison county adults ages 18 and older have not obtained a high school diploma, and 
85% of Morrison county adults ages 18 and older have not obtained a post secondary degree. Despite 
financial and educational poverty, the residents of Independent School District 482 passed a ten year $10 
million technology referendum 5 years ago. 

Over the course of five years, the school district has established its own local area and wide area network 
(LAN and WAN) for data and voice connecting all buildings within the district. A network administrator 
was hired to establish the LAN and WAN infrastructure. Internet connections and a registered domain 
name for LFCS was established. After the internet connections were in place firewall, domain servers, 
web servers, Internet mail server, proxy server and lightweight directory access protocol (LDAP) and 
dial in access were also established. 

District infrastructure expanded to include an internet drop in every classroom, a workstation on every 
teachers desk and a 1 :3 computer to student ratio throughout the district. In the fall of 1997 Little Falls 
Community Schools began an aggressive mission of moving the 1:3 ratio to a 1: 1 ratio through a laptop 
initiative. During the first year of implementation all 5th grade students had laptop computers (eMate 
300) and in the second year all 5th through 8th grade students (approximately 1 100 students) had laptop 
computers. During the first year teachers received monthly staff development training. Staff development 
for 6th through 8th grade teachers was minimal and a marked change in year two implementation was 
evident. 

Certified teachers were hired during the initial phases of the technology referendum to specialize in staff 
development and integration of technology into the existing curriculum. A district technology tool kit for 
students and staff is in place. Technology integrationists at the elementary level have established a scope 
and sequence of technology skills in relationship to reading, writing, mathematics and inquiry across all 
curricular areas. Technology integrationist function in a variety of ways through out the district. 

However, the role most closely related to district vision and goals occurs when a technology 
integrationist assists classroom instructors in the integration of technology into their curriculum by 
modeling, peer coaching and supporting educational technology in their classroom. 

Staff development opportunities are available throughout the year for teachers. Technology integrationist 
are the instructors for these classes. In addition technology integrationist are involved in various teaching 
and learning committees as well as graduation standards committees. After district scope and sequence is 
revised and aligned with graduation standards, the technology integrationist begins seeking additional 
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resources to enhance and reinforce the curricular goals. The integrationist creates and provide sample 
lessons aligning with goals and then assists the classroom teacher in implementation of the lessons. 



Little Falls Community Schools has a variety of special projects in place as a result of technological 
infrastructure, integration of technology into the curriculum, local history and community support. Please 
go to the following sites to gain information about the Fresco Project, Riverwatch Project, Foreign 
Exchange Teacher Program, Camp Ripley Job Shadowing Program and the Laptop Initiative. 
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Mantua Elementary School 

Virginia 

Mantua Elementary School-Mantua Center, located in Fairfax County, Virginia, is home to a diverse 
student population of 820 general education, deaf, English as a second language, gifted, and learning 
disabled students. The school encompasses three connected school programs-the Total Communications 
Center for the Deaf, the Gifted and Talented Center, and the base school. 

Mantua A Ba.ic School Powered by Technology integrates a technology-rich, interdisciplinary 
environment within the framework of the Basic School, the educational philosophy of the late Dr. Ernest 
Boyer. 

At Mantua, technology is viewed not as an end in itself, but rather as a tool that augments the following 
four pillars of the Basic School: 

• the School as Community (bringing into focus how people relate to one another and work 
cooperatively to solve problems), 

• a Cun'iculum with Coherence (bringing an interdisciplinary approach to the acquisition of 
knowledge), 

• a Climate for Learning (providing the physical and motivational factors necessary for effective 
teaching and learning), and 

• a Commitment to Character (emphasizing how the school experience shapes the ethical and moral 
lives of children) 

Students and staff use technology to make connections across disciplines and to integrate and apply 
literacy skills in language, mathematics, and the arts. The Basic School accommodates many learning 
styles and theories and involves parents and other community members in the learning process. Rather 
than focusing on a specific learning paradigm, our teachers select to integrate coherently the methods that 
are most appropriate for the objective at hand, while providing an environment that supports an effective 
education for every child. Technology is one tool that empowers this acquisition of knowledge; students 
use technology to simplify, facilitate, and enhance the learning process. 

The curriculum of the Mantua Basic School includes all of the traditional fields of elementary study. The 
eight commonalties of the Basic School are unique lenses through which we view the traditional 
disciplines and create interdisciplinary connections. Teachers are seen as leaders, facilitators, and 
mentors, well grounded in the art of teaching and well trained in the use of the most current computing 
equipment and software applications. Children exposed to interdisciplinary units of study arc becoming 
literate, cooperative, problem solving, self-motivated learners and that is what Mantua is about. What 
most distinguishes education at Mantua Elementary— Mantua Center is that our students are not passive 
recipients of knowledge, but rather, active participants in the full educational process. 
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Since the Global Access initiative began in 1994, the Department of Educational Accountability and the 
Office of Global Access Technology have worked collaboratively to evaluate the implementation and 
effectiveness of various aspects of the program. 

During the past five years, the emphasis of the evaluation has changed as new questions arose. The 
evaluation has had three phases. 

Phase 1: Focus on Implementation 

• Was the Global Access project implemented as planned? 

• What factors determined the extent to which the program was implemented the first year? 

Phase 11: Focus on Use 

• What were the computer applications and telecommunications resources that teachers and students 
were using? 

• Were teachers using technology for administrative functions, planning and presenting instructional 
material, and incorporating technology into classroom activities with students? 

• Were teachers beginning to use multiple applications in all three areas? 

• Were teachers beginning to change their teaching practices as a result of the capabilities of 
technology resources? 

Phase 111: Focus on Instructional Integration with Students 

• Given that teachers are technology users, which models of staff development are most effective in 
promoting technology integration into the curriculum? . 

• What are the conditions (school, content area/department, school system) that promote or hinder 
the effective use of technology to promote student learning? 

• What are appropriate methods that can be used to evaluate the effect of technology resources on 
student learning? 



-###- 
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TLCF Resource Usage 

• each of the six tech centers will include 4-6 computers, TVs VCRs, a scanner, a printer and other 
technology equipment 

• several sets of portable student computers 

• combine district and TLCF funds to install one computer in each classroom 

• staff development 

District Goals 

• to establish one multimedia technology center for each of the building's six interdisciplinary 
academic teams (150 students to each team) 

• to increase proficiency test scores 

• to improve communication among students, teachers and parents 

• to make student learning better 

Indicators of increased learning 

• evaluation criteria for student work with technology developed by district personnel in conjunction 
with the Ohio SchoolNet Lesson Labs 

• improved proficiency test scores 

• extended student electronic portfolios that are assessed by teacher-developed criteria 

Local involvement 

• open facilities for use by parents and community members 

• further existing mentor relationships with local businesses and develop new ones 

• develop virtual partnerships with cultural institutions, community resources and government 
officials 

Professional development for teachers 

Middletown City's philosophy on teacher training is that teachers learn best from teachers, especially 
those with whom they work closely; a core group of trainers that include two teachers from each of the 
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six academic teams and specialized teachers (e.g., health, phys. ed., special ed.) will participate in the 
initial training. These teachers will participate in weekly technology training comprised of basic 
computer/technology skills, how to operate certain software and how to integrate the technology 
resources provide by TLCF funding into the daily curriculum. They will then be responsible for teaching 
their interdisciplinary team colleagues what they've learned. 
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Milwaukee Public Schools (MPS) is in the midst of major high standards and reform initiatives. New 
high school graduation requirements, middle school proficiencies, and soon to be developed elementary 
school proficiencies, have generated a sense of urgency and momentum surrounding student achievement 
and school accountability. MPS is using a standards based, rigorous curriculum, with an assessment 
package that includes state standardized tests, district performance based assessments, portfolios, and 
classroom based projects. These and other changes in urban education, require new and innovative 
teaching strategies and the use of technology for instruction, assessment, and management of student 
information. 



There has been a tremendous mobilization of energy and resources to prepare staff and students to meet 
rigorous high standards. The focus is on student-centered learning environments that use technology to 
Support student's individual needs and capabilities. Changes in the curriculum that stress higher order 
and critical thinking emphasize the use of technology as a tool for teaching and learning rather than an 
entity in and of itself. 

Technology support strategies include infrastructure requirements, decentralized management, school 
accountability, professional development, collaborations, and fiscal support. The district Technology 
Strategic Plan delineates the process to provide every school in our district access and connectivity. 
Decentralized decision making and budgeting allow all schools a voice in how rapidly they acquire 
hardware, software, and professional development. Clearly delineated academic outcomes and targets 
with appropriate assessments provide the incentive for a high level of accountability. Partnerships within 
the community 7 including businesses, community based organizations, foundations, institutions of higher 
learning, and grant funding, have supported the continual use of technology across the curriculum, across 
settings, and as a vehicle to extend and enrich the learning experience. 

The MPS Technology Plan makes it possible to facilitate learning environments, anytime anywhere, with 
anybody. Various projects have utilized this philosophy, including Technology Literacy Challenge Fund 
(TLCF) professional development, the Connected Community of Learners (CCL) project, and Goals 
2000 Grant. Teachers employing technological tools for distance learning and on line instruction, have 
brought about the kind of teacher leadership that has resulted in high level student projects, teacher 
collaboration, integrated clinical approaches, and sophisticated approaches to assessment. 



The Replicable Schools Program is an important model for the integration of technology at the classroom 
level. Like the TLCF initiative, teachers receive ongoing training and support and are given the 
opportunity to network, model, and train other teachers. Both programs are examples of the systemic 
commitment to meaningful professional development and the direct connection to the classroom. The 
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focus on academic achievement, efforts to increase access, researched based decisions, and extensive 
professional development, are the key elements of Milwaukee's technology implementation. 
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PURPOSE: 

To develop teaching teams who can train other teachers to use the Internet as a curriculum resource. 

DESCRIPTION OF PROJECT: 

Interested candidates met at an information session to take a pre-test and see models of the products they 
would be asked to create. Two participants from 44 sites (including 8 private and parochial schools and 
the Public Library), where one participant was the school LMS, were selected for four days of training 
the first year, and five days the second year. Through training at our Regional Educational Service 
agency for the county, ACES, teams of teachers and library media specialists have created a professional 
development website, including a curriculum-related project that utilizes resources from the World Wide 
Web. Projects demonstrate participants' experience with Internet search strategies, the identification and 
assessment of appropriate curriculum-related websites, and website development. Each team trained 
other teachers at their schools, using their website projects, which are published on the Web through the 
use of FrontPage98. Through a strong focus on the integration of the Internet into the curriculum, 
participants earned many other computer skills, such as word-processing, creating and utilizing electronic 
images, desktop publishing and multi-media integration. 

GOALS for the second year of training (in terms of Expectations) 

At the end of the 5 sessions, participants will: 

• Develop search strategies on the Internet. 

• Demonstrate how to identify and assess curriculum-related websites. 

• Create a website with FrontPage98 that can be a useful tool for professional development. 

• Select and publish a list of curriculum related websites for one or more teachers in the school (as 
part of their professional development tool). 

• Create and publish on the WWW a curriculum project with FrontPage98. 

• Use what they have learned and created to provide professional development to members of a 
school staff. 

-###- 
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In the fall of 1993 the Norman Public Schools formed a 28 member Technology Planning Committee. 
Membership included parents, members of the community, teachers and administrators. Issues addressed 
included instruction, staff development, management and funding. The Committee developed a vision for 
technology and goals for the implementation of the vision: 

Vision statements were included for "Technology and Learning", "Technology and Teaching", 
"Technology and Assessment", "Technology and the Community", and "Technology and the Work of All 
District Staff'. 

Goals include: 

1 . Establish a computer communications network for the purpose of information access and sharing 
among classrooms within a school and among schools within the district, with parents and the 
community, and among district personnel. 

2. Establish a video production/delivery network for the purpose of instruction, information sharing 
and information access. 

3 . Assure equity of access, opportunity and experience for all teachers, students, parents and 
members of the community through technology. 

4. Establish the library media center as the technology hub of each 

Instructional site. 

Since 1994 a great deal of progress has taken place: 

Training: Staff development is key in integrating technology into the curriculum. The Norman district 
has developed a cadre of teachers called Pro Teachers for Technology, obtained grants to develop and 
staff a Technology Training Lab, and developed NPS Technology Day which has become a statewide 
professional development conference. Sites develop technology goals as a part of their site plans for 
school improvement and plan technology training. A full schedule of technology training is offered by 
the district focusing upon integration of technology in the classroom and stipends are paid to teachers for 
summer participation. 

Access to Hardware and Software: Bond funds, Title I funds, PTA contributions, various grants and 
the district general fund have supported access to technology in this district which operates with less than 
the state average in per pupil expenditure. Access to computers has more than tripled; instructional 
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software, including integrated learning systems at some sites, is in place. A "Unified Desktop" design 
(district standard) provides password entry to a system that combines a full PC workstation with the 
capabilities of network computers. 



Connectivity: A wide area network begun in December of 1995 (fiber provided by the cable television 
company) now connects every school and LANs connect almost every classroom - will connect every 
classroom by the fall of 1999. 

Technology Support: Standardization of software and hardware, extended warranties, outsourcing 
contracts, and a team approach to technology support were put in place in 1998 to facilitate efficient 
technology support. The team approach combines the resources and abilities of the Information Systems 
and the Instructional Technology departments. 



Although the district technology plan has been updated since in was put in place in 1994, work on a new 
plan with a more focused instructional emphasis will begin in the fall of 1999. 

-###- 
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In November, 1992, the Ohio County Schools Technology Implementation Committee was charged with 
studying the current and future technological needs of the school system, as well as developing an 
implementation plan for the effective use of technology in instruction and administration. 

This charge resulted in the development of "Technology: Our Future Now - A Five Year Plan and 
Beyond". This initial technology plan served as the basis for implementing the hardware, software and 
infrastructure throughout the system. 

The funding for this project was realized through a bond levy that was approved by more than 70% of the 
voters. The initial implementation was phased in over a five-year period. Some of the highlights of this 
implementation are as follows: 

• appointment of a district technology coordinator 

• continuation of the technology committee 

• designation of building technology specialists 

• infrastructure development (cabling plan for all classrooms) 

• placement of four workstations and a printer connected to the school-wide network in all 
classrooms 

• implementation of WVEIS (West Virginia Education Information System) for administrative 
management 

• provide for staff training (instructional and administrative) 

• replacement of existing phone system, providing for voice processing at all locations 

• installation of FAX machines in each building 

• purchase of appropriate software 

As the technology plan was a dynamic document, it was revised and updated by the committee. In 
addition to the original implementation, the Bell Atlantic World School Project was supported. This 
project provided internet access to all schools via a 56kB Cisco router. Two other statewide projects also 
were implemented. The Governor's Basic Skills Program provides for software and hardware in grades 
K-6. The WV-SUCCESS (Student Utilization of Computers in Curriculum for the Enhancement of 
Scholastic Skills) is now in its third year. This project provides internet computers in grades 7-12, along 
with office suite and career exploration software. 

In addition, Ohio County is actively involved in local, state and federal grant programs. Examples of 
these include: 

• CTRC program, which has provided multimedia laptops and training 
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• Telecommunications/Technology grants that have provided additional internet computers 

• Infomine Project for accessing library databases 

• TLCF (Technology Literacy Challenge Fund) Grants 

• Partnership with local TV station to provide for an internet weather station 

• School Technology Team training 

• Partnerships with local colleges and universities to provide staff development 

• Phase 9 Training through the High Tech Consortium in Fairmont 



1 1 II J i 
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The Okaloosa School District has a comprehensive plan for technology which includes technology 
infrastructure, teacher training, access for students and instructional software. The following is an 
overview of these areas of technology: 

The District has an Okaloosa Metropolitan Area Network (OMAN) and each school has either a T1 or 
56KB line We have two T1 lines for Internet access, one provided by the State of Florida FIRN network. 
The District also has a BESS Proxy Server from N2FI2 that filters unwanted Internet sites. All schools 
have as a minimum 10 Internet lines in either the media center or a lab. This gives access to the Internet 
for all students. Many schools are completely retrofitted for voice, video and data. During the next 5 
years, the rest of the schools will be retrofitted. 

The State of Florida has developed Sunshine State Standards for students. These standards ha\e 
technology woven throughout them. The District selected Student Standards for Technology from the 
Sunshine State Standards. Based on the student standards, the District developed Basic Techno ogy 
Competencies for Teachers. This is what teachers need to know in order to teach the students. Training 
for basic competencies if provided through school-based mentors, District trainers, an ot er mo es o 
training. The District has developed a model for training teachers to integrate technology into the 
curriculum, TOOLS 2000. In this workshop teachers leam basic computer skills, use of a digital camera, 
instruction on the writing process, technology based activity centers, theory of multiple intelligence, 
authentic assessment using a rubric, managing a classroom with technology and p aiming t ema ic uni s. 
The workshop has been very successful and is now being piloted online for teachers to take from sc oo 
or home The District has implemented an incentive program called Top Notch Teacher. When teachers 
complete basic competencies, take a test on them, and show by means of a portfolio that they are 
integrating technology into the curriculum, they receive three new computers for their classroom. 

Through grants and State Technology funds the Okaloosa School District and schools are purchasing 
instructional software programs for students. This year two elementary schools received a TLCF grant 
for reading. Computer Curriculum Corporation reading software and Lightspan Home School programs 
were purchased. One of our high schools received a TLCF grant to implement a two-year Algebra I c ass 
to assist students who normally don't take Algebra. The State recently required that all students complete 
Algebra I. The District also received a TLCF grant for distance learning and instructional television, Best 
Practices in Action. In this grant, a tower is being constructed in Bay County that will connect all the 
counties in the Panhandle for instructional television, put the TOOLS 2000 training online and produce 
training and parent awareness videos for technology. 

As you can see, there is a lot going on in the Okaloosa School District. We are proud of our 
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accomplishments and are looking forward to the future with technology as a tool for learning. 
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Short Description 

The Oswego school district's technology program has accomplished what others have failed to do. It 
enhances teaching and learning through an enterprise-wide system that includes a reliable network, 
strong staff development, and web-enabled instmctional materials, assessment, and training. 

Long Description 

Earlier this decade, school districts around the nation began hearing about the power and promise of 
using information technology to improve teaching and learning. Many went on a buying spree, cobbling 
together the public and private resources necessary to wire their schools for connectivity to the Internet 
and to equip classrooms with computer terminals. 

Today, many of these districts have unprecedented access to technology. Most, however, have not 
demonstrated that they have the means to use the technology to improve learning. As a result, 95 percent 
of the instructional technology programs implemented in districts have not lasted more than three to five 
years, studies show. 

Oswego, with 415 teachers and 5,600 students - nearly 40 percent of whom live in poverty - is different. 
Despite a shrinking property tax base, the district has been able to sustain and build an impressive 
enterprise-wide information delivery system that fully integrates a network, instructional and 
communication materials into teaching and learning. Further, the system is: 1) used and understood by 
teachers and students; 2) continually updated to avoid obsolescence; 3) driven by the needs of teachers 
and students, not by the needs and desires of outside vendors; and 4) fully integrates a network, 
instructional materials 

The district owes much of its success to careful planning. Drawing on the recommendations of teachers, 
parents, school board members, business people and community leaders, Oswego has developed and 
implemented a five-year strategic plan that has helped turn skeptics and technophobes into technophiles. 
Following are the major features of the system: 

To ensure that teachers understand how to use the technology, Oswego equips a classroom with 
computers only after the classroom's teacher participates in extensive training. The training includes 27 
hours of instruction on Internet access and 30 hours on basic computer skills. Only a handful of the 
district's 415 teachers have not participated. The average teacher has taken more than 130 hours of 
instmction; in some schools teachers have taken as many as 250 hours. More than 50 courses are offered. 
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And because most software manuals are written for business users, the district customizes and writes its 
own manuals so that examples are relevant to the classroom. 

Oswego commits 2 percent of its budget to support technology each year. The investment allows Oswego 
schools to replace 15 to 20 percent of its equipment annually, thereby preventing obsolescence. In 
addition, the district has received hundreds of thousands of dollars from private sources and vendors 
through donations and in-kind services. 

The district has two full-time technicians who staff a central help desk. In most cases they are able to 
diagnose and correct problems remotely over the network, obviating the need to travel to schools and 
interrupt classroom instruction, or the need to turn teachers into computer technicians, allowing teachers 
to focus on instructional integration. Redundant systems keep the system running, even when pieces of it 
shut down. 

Oswego uses a district-wide area network (WAN) as a backbone for its technology infrastructure. The 
network links all district buildings to each other and to the Internet. Fiber-optics are used to distribute 
data to the buildings, guaranteeing high bandwidth and fast throughput and allowing rapid access to 
network applications, and internal and external information sources. Within the buildings, data is 
distributed to the classroom over category 5 cable at a rate of 100 megabits per second. 

Each classroom is equipped with four student workstations, one teacher workstation, and printing and 
classroom-wide viewing capabilities. The configuration assures all students relatively easy access to the 
Web. In all, 2,000 Compaq PCs in eight separate school houses, a district office, a transportation center 
and a warehouse are linked by a high performance network of Compaq computer servers, supported. by 
Windows NT server software. 

Because a system is only as good as the information it carries, Oswego spent considerable time and effort 
surveying other model districts. The research helped the district select an extraordinary array of 
instructional resources that it delivers to every desktop of students, teachers, staff and the community. 

At the click of a mouse, students can access local library collections, a huge volume of online resource 
materials and a number of different research and curriculum applications to support teaching and 
learning. Teachers can share information on student assignments, curriculum resources and professional 
enrichment activities. They can link to technology tools and manuals and use e-mail to communicate 
with their colleagues and students. 

Many of the same resources are available to parents and community members. They also can review' 
student performance data, lunch menus, calendars, news about the district and academic standards. 

In sum, many districts aspire to build a global schoolhouse; Oswego has succeeded. It is no wonder then, 
why 67 school districts from around the U.S. and Asia visited the Oswego City School District 
Technology Program during the 1997-98 school year alone. 

Benefits 

Anyone who logs on to a personal computer in an Oswego classroom, kindergarten through 12the grade, 
cannot help but recognize how technology has fundamentally redefined the way students leam and 
teachers teach in the district. 
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For all K- 12 students, the desktop allows access to the following: 

' • All the resources of their school and city libraries 

• A periodical database that covers 800 online publications, including Congressional Quarterly, an 
on-line dictionary and thesaurus and six different encyclopedias — World Book On-Line, Encarta 
98, American, Grolier, Compton's Interactive and the Ultimate Children’s Encyclopedia; 

• Access to topic-related databases available through several commercial providers, including 
Electric Library, SIRS Researcher and ProQuest; 

• Access to Web search engines, including Yahoo, AltaVista, Excite, HotBot, InfoSeek, Lycos, and 
N Light 

• A number of newswires and network news services, including AP, Reuters, Bloomberg, ABC, 

CBS, CNN, Fox, MSNBC, NPR, ESPN and the Weather Channel 

• Regional and national newspapers, domestic and international magazines arranged by subject or 
country of origin, and domestic and international trade publications 

Students have at their fingertips such tools as Microsoft Office, Publisher, Adobe Photoshop, LogalMath 
Science and Computer Curriculum Corporation's Video intensive instructional Learning System. 

In addition, students can access GaleNet, a comprehensive database that includes: 440,000 nonprofit 
membership associations, as well as brochures, and descriptive materials for 2,500 of these 
organizations; 282,000 consumer brands and 51,000 companies that manufacture them; complete 
biographical and references and information on 100,000 contemporary U.S. and international authors; 
critical essays on contemporary authors; student-focused materials on most studied authors, biographies, 
science and history topics and multicultural issues; student-focused materials on Shakespeare, poetry and 
world history; several of the most popular college guides; and a database of publications and broadcast 
media. 

Beyond the rich list of library resources, students can take grade- and subject-specific practice quizzes 
and play educational games to improve their skills. In many cases, these quizzes and games are designed 
and posted by their teachers using off-site software developed at the University of Hawaii. 

Students also can connect to a vast array of off-site programs designed to improve skills. For example, 
they can link to the Math Forum, an on-line tutorial service, located at Swarthmore College, for students 
in elementary, secondary and post-secondary schools. They also can link to National Geographic, Cyber 
Seuss, the Electric Zoo and other on-line services that educate and entertain at the same time. 

Students can post and view PowerPoint projects developed by them and their classmates for classroom 
credit. The projects are posted by student name, grade and topic. Subjects covered in the presentations 
include everything from African art to movie reviews and Shaq O'Neill. 

The district is currently developing a site that will offer students study guides and materials to prepare 
them for the New York State Board of Regents exam. As is the case with many locations on the Oswego 
desktop, this site will include material developed by teachers as well as students. For example, as part of 
a class project, students already have developed a site called "Great Cases of the Supreme Court," which 
will help prepare readers for the government section of the Regents exam. Another site developed by a 
technology consultant who works for the district includes a subject-specific bibliography of teacher 
resources, student activities and references that will help prepare students for the Regents exam in 
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chemistry. 

For students as well as teachers, a technology section of the desktop contains a variety of Web site 
development tools, including tips, tutorials and multimedia files. Examples of sites developed by students 
are also posted. The technology section gives teachers an up-to-date list of available in-service 
professional development courses to improve their technology skills. 

Teachers can review curriculum materials posted on the site and, using a search engine, hunt for these 
materials by subject area, resource type and grade level. 

It is clear from viewing the desktop that the level of teacher participation is deep. One page lists dozens 
of subject guides developed by Oswego teachers and posted for their colleagues and for students. They 
include a step-by-step guide for writing research papers, guides to literary terminology, roots and 
prefixes, parts of speech, composers, trinomial factors, Spanish and French vocabulary lessons. 

Through an on-line service called CCCnet, teachers also can access outside resources for up-to-date 
curriculum materials, mentors and subject-specific topics. CCCnet also offers students study materials in 
math, science, social studies, reading and language arts. 

These resources, and more, have changed how students work and demonstrate their competencies. In 
their subject areas they have ample opportunity to show what they know with games, quizzes and 
projects. In addition, students master the fundamentals of making presentations with PowerPoint, 
creating text documents with Word and building relational databases with Excel. They use such powerful 
desktop software as Microsoft Publisher, Adobie Illustrator, PageMaker, Photoshop and Acrobat. More 
than 65,000 clipart images are available to students through various databases. 

Because students have high bandwith access from workstations, they essentially replace six- or 
seven-year-old textbooks with virtual textbooks that can be updated with a stroke of the keyboard. 

For teachers, the technology solutions are changing not only how they teach, but the way they 
communicate with each other. Until recently, teachers in Oswego were no different from teachers 
anywhere else: They tended to be isolated from their peers. The new technology, however, gives them 
opportunities to share information and promising practices not only with other teachers in their building, 
but also with teachers elsewhere in the district, the state, the nation or the world. They can take ideas 
posted on their desktop and implement them as they are, or change them to suit their teaching style or 
their students' needs. 



Importance 

Information technology has changed the business of education by placing entire libraries of material at 
students' fingertips. Students who otherwise might be turned off by material in a book eagerly follow 
leads and links that allow them to leap from one island of information to another, coupling the challenge 
of a puzzle with the joy of discovery. 

Similarly, students who might be loath to pick up a pen to write an essay can become authors, artists, and 
animators using word processors, graphics software and presentation programs to do their projects. This 
can be demonstrated by a high school student who won the ESPY Writing Award in 1997 and attributed 
his success to the technology located in the high school writing center. 
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But just as the young seem naturally inclined to embrace the new technology, their teachers sometimes 
do not share this enthusiasm and lose interest in working with a tool for which they were not properly 
trained. 

Oswego's goals were: 

• Bring teachers up to speed on the new medium and keep them there 

• Migrate learning from paper and blackboards to electronic media 

• Develop a system that would serve as the students' guide to help them find what they seek 

This last point is crucial because while students may initially enjoy working on a computer, it can 
become a frustrating experience if they are unable to find the material they need to do their work. That is 
why the school district's network is laden with pointers, tips, material, links, search engines, resource 
materials, and database access. 

The network provides a similar range of resources to help supply teachers with the tools to do their work 
and to allow them to communicate with each other. The network's resources receive constant attention 
and updating. It is not treated as an afterthought. 

Oswego recognizes this as the tool that it is, and everything about it is directed toward the purpose of 
education. 

Originality 

The Global Schoolhouse was conceived and built as an integrated system to serve the educational needs 
of its teachers and the learning styles of its students. It was not jury-rigged to fit around existing pieces. 
The network in Oswego is built to accommodate the needs of educators, and the pieces that rely on the 
network were designed to work with it. For example: 

• The network hardware was designed to connect remote campuses with high-speed, redundant 
systems to ensure reliability and convenience. 

• Teachers have strong incentive to train themselves The training teachers and staff receive assures 
that they will use the system wisely and efficiently. 

• The manuals teachers and staff use in their training cite examples that resonate with educators. 
They describe what teachers will experience in the classroom and what they will experience on the 
network. 

• The educational software delivered to students works well on the network, and serves the 
instmctional needs of teachers as well as the educational needs of students. 

The way students use the technology is different from the way it's used in many districts. Oswego has 
established what it calls "electronic learning environments" - classrooms where every student is 
equipped with a desktop computer. The learning that occurs in these rooms is completely linked to 
instructional technology. 

The manner in which Oswego uses its resources also is different. From the very beginning, the district 
recognized that it did not have money to bum, time to lose, or human resources to squander. Oswego 
bought what it had to, foraged for good ideas from other institutions, and scrounged for quality material 
that is available free over the Internet. The district established partnerships, benefited from' corporate 
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generosity, and did a lot of legwork on its own. It used others' partial solutions to create a whole system. 



For example, teachers avail themselves of free online quiz makers so their students can take customized, 
electronic tests. Teachers put up their own Web pages to give students tips on topics, such as how to 
write a research paper. The district planned carefully, shopped wisely, and worked hard. Teachers were 
brought into the process early and continue to spearhead the effort. 

The result, in the end, is one integrated system that helps the teacher provide enhanced learning 
opportunities for students. 



Success 

The Oswego district has met and exceeded its goals for professional development and use of the network 
by teachers, students and the Oswego community in general. Outside evaluators have concluded that the 
district has accomplished in one year what many expected would take three years. 

The district set a first-year goal of reaching 35 percent of its faculty and staff in professional 
development programs for integrating technology into the classroom. Instead, 85 percent enrolled. As of 
today, 99.6 percent of teachers and staff have taken the requisite number of courses needed to qualify for 
technology access. Four staff development rooms, each seating 40 teachers and operating four nights a 
week, are filled to capacity. 

High volume use of the system is evident. The T1 Internet connection is so saturated that the district is 
now evaluating the use of a segmented T3 connection. Put another way, the district currently has 1,200 of 
its 2,000 computers active during any given moment of the school day. Since June 1998, more than 
36,500 have visited the district's Web site, which suggests heavy use by the district as well as people 
around the world. 

Use of the system is expected to double each year over the next three to five years. In many districts, 
where small LANs and numerous administrators are the norm, such growth would be problematic. But 
Oswego, with its Windows NT network and centrally-located servers, is ready to support the expansion. 

Elementary school students in Oswego are miles ahead of children their age group in technology skills. 
First and second grade students are using electronic resources to research thematic units and 
communicating what they've learned through a Microsoft PowerPoint presentation or the creation of a 
Web site. 

Students interested in drafting or engineering careers are becoming adept at AutoDesk computer-assisted 
design, drafting and manufacturing. Those who take three sequences of AudoDesk courses are qualified 
to take the AudoDesk licensing exam. If they pass, they can compete for jobs earning $35,000 a year. 
Similarly, the exposure of high school students to the Microsoft Office suite is helping some pass the 
performance exams for Microsoft certification. 

The district's digitally equipped television studio is giving students hands-on training in communications 
fields. William Bellow, an instructor in the communications studio, says 22 students who graduated from 
his course last year were accepted into college communication programs. "Ninety percent of these kids 
ultimately are graduating from the college communication programs they enter," he says. Bellow says 
some of the more competent graduates are anchoring local television news shows or are working in the 
Learning Company and The Children's Television Workshop. Oswego Communication students leave 
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competent in industry standard equipment and software such as: 3D-Max, Alladin, Character Studio, 
Inscriber CG, non-linear editing, digital cameras/decks and telescript. Oswego also boast the only 
Regents approved Communications Major in new York State with all of its college-bound students 
entering communications programs in some form of advanced standing. 



Examples abound of teachers who are personally benefiting from the technology focus. Thomas Caswell, 
a third-year social studies teacher at Oswego's high school, came to the district with no special skills in 
technology - he didn't acquire his own personal computer until 1997. Today, using distance learning, he 
is earning a master's degree in instructional technology. He is one of 25 educators in the country 
nominated and recognized for his exemplary integration of technology into the curriculum. 



Julie Burger, a second grade school teacher, has completed nearly 200 hours of inservice technology 
courses. Her students create reports using Microsoft Word, digital cameras and the Internet. "Students in 
my class have typically been successful," she says. "I never realized though how much deeper their 
understanding and knowledge could be until I started using technology in my class. There has been a 
dramatic change in how much better and faster my students leam with our technology." 

Outside organizations have recognized Oswego's progress. Microsoft cited Oswego in 1997 as a creative 
and innovative school. Digital Equipment Corporation gave the district its Master of Innovation award 
this year. And Compaq Computers says Oswego is in the top 5 percent of districts in the country that 
have integrated technology into the curriculum. 

"Our close association with the Oswego School District has provided us with a solid example of how 
leading-edge technology, backed by vision and drive, can produce compelling results," says Susan 
Twombly. "By augmenting traditional teaching methods with the practical application of the latest 
technology, they are truly innovators in education." 

The district plans within the next two years to add five more elements to the technology mix: 1) access to 
curriculum servers from teachers' and students' homes and remote locations either by cable television or 
modem dialup; 2) a video distribution system; 3) video on demand to support instruction; 4) electronic 
lesson plan files; and 5) a Windows-based student accounting and record-keeping system. 



Difficulty 

School districts typically encounter the following problems: they put computers into classrooms without 
showing teachers how to use them to improve learning; they plan for today, rather than the future, and 
they rush to judgment on technology solutions to satisfy the political demands of policymakers who need 
to assure the public that the district is doing something. 

Kenneth W. Eastwood, Oswego's assistant superintendent for secondary education and technology, says 
his district initially fell into the same traps as other school systems. In 1991, after substantial investment, 
its teachers and students had little access to computer labs, teachers had little or no training in how to use 
the equipment, and few had figured out how to make computers relevant to their classrooms. As much as 
85 percent of the technology was already obsolete, and no long-range plan existed for sustaining the 
technology effort. 

Oswego began reversing its course when it decided to commit to a strategic plan that would make 
technology the slave to instruction, and not the other way around. "Schools typically get the boxes, then 
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do the planning," Eastwood says. "We identified the needs, solutions and then talked about the 
technology before we spent a nickel on it." 

The strategic planning process, which began in 1994, identified the expectations of parents, teachers 
students and the local business community. Further, to assure that its technology plans truly suited 
teachers' instructional needs, the district took information it gathered from interviews and used it to 
develop a survey instrument to poll teachers on classroom needs. Responses were received from 95 
percent of them. 

The planning process helped create a critical mass of support for technology in the classroom. It also 
helped define the infrastructure the school would use. And because so many people were consulted, the 
process helped create buy-in for funding from critical stakeholders. 

Nevertheless, in some school buildings principals emerged as obstacles to technology because of a 
combination of their age and lack of exposure. The district opted to go directly to the customer - the 
teachers. It told teachers they could act on their own to acquire instructional technology in their 
classrooms; all they had to do was take the requisite number of professional development courses. 



As a result of these strategies, the district was able to overcome resistance and obstacles. Today most 
residents of Oswego are proud of their schools' technology system and are users of it. 



u u {A 
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South Carolina 



Richland School District One covers 482 square miles and serves the capital city of Columbia, South 
Carolina encompassing urban, suburban and rural neighborhoods. A diverse student enrollment of 27,000 
students in 51 schools and an equal number of adult education students, give the district its character and 
strength. 

Richland School District One has embarked on a significant, aggressive plan to acquire, support and use 
technology throughout schools and the community. The establishment of the Information Resource 
Management Department in 1995, combining and expanding the former Educational Technology , Media 
Services, and Electronic Repair Units, signaled the beginning of the district's focus on the integration of 
technology in the instructional arena. 

The successful 184 million-dollar bond referendum in 1996 enabled the district to support new school 
construction, school renovation and renewal and the establishment and expansion of the district wide 
technology infrastructure. A three year, four-pronged infrastructure plan, funded by the bond at 9.23 
million dollars, was initiated in the spring of 1997 that: 

1 . created the wide area network with Internet access at all locations 

2. expanded all school LANs to all classrooms 

3. replaced file servers in every school 

4. placed two additional computers in every classroom and 15 computers in every media center. 

Coupled with these initiatives was the expansion of the technology education program. Courses, modules 
and workshops offered throughout the school year and summer months are keyed to the district's 
technology competencies that all employees are expected to master. District-level instructors are 
full-time technology educators experienced in technology training and integration. Classroom teachers 
who have demonstrated skill in using technology effectively provide training for evening and summer 
courses. Additionally, media specialists provide training during and after the school day. 

Supplementing the district's technology implementation is a five-year, 4.3 million-dollar Technology 
Innovation Challenge Grant awarded in October 1997. The grant, titled Richland Clicks! focuses on 
improving student achievement and community involvement through technology. A wide array of grant 
activities provides a comprehensive approach to using technology as a tool. An instructional website, 
curriculum connections lessons, laptop lending program, technology van, and strong community 
partnerships define the scope of the grant. 

Richland One is a beneficiary of state-level initiatives focused on enabling school districts to move 
forward with successful technology programs. Funded by the state legislature, and administered under 
the direction of the State Department of Education, millions of dollars have been appropriated to 
establish infrastructure, Internet and telecommunications, purchase hardware and software, and support 
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the professional development needs of educators. A recently revised State Technology Plan, curriculum 
frameworks, achievement standards and regional technology centers increase the depth of resources to 
meet local needs. 



Much has been accomplished in Richland One in a few short years. Preparing our students, parents and 
community for the future will continue to challenge our skills and creativity. 
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New Mexico 



The Roy EPSS (Educational Plan for Student Success), a New Mexico Department of Education 
mandate, is an integral, yet changing part of the Roy educational system, technology in particular. New 
Mexico EPSS is an attempt to align local and state standards and benchmarks with national education 
goals including: CITE -- Consolidating Initiatives for Tomorrow's Education, New Mexico Standards of 
Excellence, President's Educational Technology Initiative, and Roadmap to School Improvement. 
Everyone is involved in the Roy community, students, businesses, faculty, administration, and families. 
Everyplan, action, activity, and evaluation is tied to our EPSS. 

The turning point for our technology plan was the implementation of the EPSS in 1995 which proved to 
be the driving force in the integration of technology into the curriculum. Prior to 1995, there were several 
computers and printers in place. However, there was not a long range strategic plan providing for focus 
and direction. We lacked qualified personnel with the experience and knowledge necessary to lead our 
school into the technological millennium. The administration and faculty through community input 
realized that although we had the hardware, something was missing. Parents, teachers and students 
* wanted to be able to use technology. 

Technology, at this time, was limited to basic computer games and word processing with high schools 
students having primary access. The school's resources were not adequate to address the technological 
needs of the Roy students. The success in acquiring grant funding and a clear vision has enabled our 
school to provide state of the art technology including ACAD, Multimedia studios and labs, voice 
recognition capabilities, networking, 21 CD ROM tower, schoolwide computerized grading system, 
individualized education and career planning, website design, and Internet explorations. 

To insure the success of technology across all subject areas and grade levels, an integrated technology 
curriculum was written by students and teachers. As a result, technology has become the norm by 
everyday use of hardware and software available. Integration of technology has consolidated the efforts 
of teachers and students to result in outstanding projects satisfying academic requirements of more than 
one teacher across curriculum. 



Our system was inadequate to meet the demands of the students, faculty, and community needs. 
Additional grant moneys were awarded to implement programs to satisfy students needs and to allow our 
graduate to be very competitive. 



Our exceptional technology program located in a remote and mral area has grown to be an exemplary 
model for New Mexico schools. 
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S.A.D. #4 
HOMEPAGE 

The S.A.D #4 Technology Committee has met on a regular basis for the past two years in developing and 
revising, the district's comprehensive Technology Plan. 

The Committee is comprised of the Superintendent of Schools, Guilford Public Librarian, Guilford First 
Select! lan/Technology Consultant, Community Education Director, Special Education teacher, three 
classroom teachers (high school, middle school, elementary school, and three administrators (high 
school, middle school; and elementary school.) 

The Committee has conducted staff surveys to help establish priorities for staff development and 
determine levels of effectiveness in classroom integration The school board's Curriculum Committee has 
engaged in establishing goals, action strategies and evaluation guidelines. 

The S.A.D #4 plan highlights key areas of progress for our district: 

• Training 75% of the K-8 staff in utilizing technology effectively in the classroom. 

• Purchasing one hundred new computers for K-12 in the past twelve months. Establishing a fiber 
optic 5-1 2 system with T 1 access for Internet access. 

• Providing telephone, E-mail and Internet- access to all classrooms in S.A.D # 4 

• Establishing a comprehensive district Web Page. 

• Providing extensive summer and after-school computer activities for K-6 students. 

• Establishing a K-12 "data strategies" approach to managing student's continuous progress on 
Maine's Learning Results. 

• Invited to serve on the initial State of Maine Distance Learning Project 

• Selected by the State of Maine as the representative at a national conference on the "Effectiveness 
of Technology in Teaching." 

The S.A.D. #4 Technology Plan combines past experiences, current practices and a vision for the future. 

H 4(41 
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Sherman Elementary School 

West Virginia 

At Sherman Elementary School technology is incorporated in our curriculum in a number of creative and 
innovative ways. Our school has four Jostens Basic Skills Computer Labs that children in grades 
Kindergarten through Six work in daily through the departmentalization of classes. The Jostens Basic 
Skills labs promote reinforcing and reteaching of basic reading, math and writing skills. Children work at 
computer stations on their own level and at their own pace to review basic skills daily. Teachers place 
students on appropriate levels with the Basic Skills Inventory (BSI). The computer lab teachers are also 
able to track students' progress with several reports the program generates. Our Fifth/Sixth grade 
computer lab has recently been upgraded with ten new IBM computers, programmed with Compton's 
Encyclopedia, Office '97, and the Tomorrow's Promise program. This was accomplished using funds 
received from the Technology Literacy Challenge Grant and with funds from Boone County Schools. 
Sherman Elementary plans to completely upgrade this lab with additional computers purchased with 
Budget Digest Grant Funds through the office of WV Senator Lloyd Jackson. Our Preschool children 
will benefit from this grant funding by allocations for the purchase of new learning software for the 
Preschool Computer Lab. 

Sherman Elementary staff will be trained in the use of Power Point technology to enhance lessons and 
for presentations on local, state, and national levels. Plans are now also being made to purchase the 
Accelerated Reader Program through funds from grant writing sources, Business Partners and 
school/community funding. Accelerated Reader is a computerized reading program and with the STAR 
database program tracks students' progress and reading growth. This program will enhance our current 
reading program at the first through sixth grade levels and allow students to progress at their own pace. 

Children at the Fourth, Fifth and Sixth grade levels participate in our Higher Order Thinking Skills 
(H.O.T.S.) Computer Lab. The H.O.T.S. classroom is a computer lab with 13 new Internet accessible IM 
computers equipped with Compton's encyclopedias, a trained H.O.T.S. teacher and a H.O.T.S. Aide. This 
computer lab is unique in that it utilizes technology to promote positive self-concepts and to increase 
higher level thinking. The program targets priority students, but all Fifth and Sixth graders utilize the lab 
once a week. 

Sherman Elementary offers parents an opportunity to take-home computers with our Take-Home 
Notebook Computer Program. Windows 95 and the Emerging Literacy Program Training are offered 
three times a year for the notebook computer program. Parents are trained in a daylong training, on the 
use of these programs to assist their children in learning. This program is exceptional in that it was 
created by Boone County's Title 1 Director, Carolyn Miller. Another innovative program Sherman 
Elementary offers is the Lightspan Program. This program utilizes Sony PlayStations as a tool for 
learning, along with interactive software in a game format for Language Arts, Mathematics, Science and 
Writing. Family Training is provided throughout the year for parents who wish to work at home with 
their children. The PlayStations are also used in the Title 1 classrooms for instruction and reteaching. 
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Sherman Elementary' Comprehensive needs Assessment (CAN) is conducted using computer 
technology. Information gathered from our need assessment, such as standardized test scores, is logged 
and stored on the Sherman Elementary Title 1 Needs Assessment Database. This database was created by 
a member of our school community, Bruce Williams, who is principal of Whitesville Junior High School. 
Our school’s Unified School Improvement Plan highlights our Use of Technology, Technical Assistance, 
and Support Plan along with our current Technology Infrastructure Diagram. This document is updated 
yearly in conjunction with our Local School Improvement Council, Principal, Technology Team and 
staff. 
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Prior to 1994, technology was the responsibility of the school's principal. The money for technology, 
then called computers, came from the school's operating budget. Principals purchased the equipment, 
placed it in the classrooms, and the teacher decided what to do with it. 

South Harrison hired a Director of Technology in 1994. It would be the responsibility of the Director to 
develop a technology plan; purchase, install, and maintain equipment; train the staff; and create policies 
and procedures. A District Technology Committee was formed and a five-year plan created. The plan 
contained policies, procedures, criteria for hardware and software selection, and a plan for 
implementation. 

The first three years, the focus was on acquiring hardware and software. Office networks were created in 
the elementary schools and the two secondary buildings were wired to every classroom. Elementary 
classrooms received small pods of computers while secondary schools focused on lab settings. Word 
processing and content driven software was the norm. 

The third year of implementation some major adjustments were made in the approach to technology 
usage. The technology was becoming more powerful. Creation, production, and presentation software 
were emerging. Multimedia added a new dimension. Teachers and the Director of Technology were 
disenchanted with the "drill" based software. It was felt that this software was not making a difference in 
improving student learning. Students discovered ways around the software without focusing on the skill, 
or the student became frustrated when he/she could not master the skill and the computer was insensitive 
to the student's needs. The focus for software selection changed from content based to technology as a 
tool. 

With the change in focus for software and application, came a change in the methodology for 
implementation of technology. Labs, even at the secondary level, would be limited to only content areas 
that required computers such as autocad. Other labs would be dismantled and moved into classrooms. 

* In the most recent years, the focus shifted from hardware and software to connectivity and professional 
development, not that hardware and software are forgotten. Grant monies have enabled South Harrison to 
be able to continue hardware and software implementation while building the networks and providing 
professional development. The Corydon Campus schools were connected in one network. The South 
Central Campus schools were connected in another network. The next phase is to connect the two 
campuses and bring the two outlying elementary schools into one network. 

A cadre of teachers was trained to be teacher trainers. Release time, Saturdays, and summer workshops 
have been provided extensively over the last school year. Placement of mini labs in secondary 
classrooms required a change from traditional whole class instruction. The professional development 
needed to address more than "how to" manipulate the technology. The workshops were designed to assist 
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in connecting all the pieces. Curriculum, instructional strategies, alternative assessment, and classroom 
management using technology as a tool to improve student learning by engaging the learner in authentic 
project based learning became the focus of all professional development. 
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Wichita Worth County Schools 
Wichita, Kansas 



North is an urban school in a large school district. There are 1 ,600 students and 56% of them are from 
disadvantaged homes. The school is culturally diverse with 3% Indian, 6% Asian, 19% African 
American, 31% Hispanic, and 42% white students. 

Two years ago I was hired to bring technology to the students who use the school's library media center. 
Many of the students at North do not have access to computers at home so they are at a disadvantage 
when they leave North and try to compete with more advantaged students at the universities and in 
careers. 

Two years ago I started planning with district personnel for a 60 computer networked technology lab. I 
had a 5 year plan to accomplish this goal. During the first year the infrastructure for the network was put 
in place. This took a lot of patience and money to accomplish. We installed a server with NT software, 

15 client computers, a 14 port CD server, and a cable modem for Internet connectivity. During the 1st 
year the students and staff at North were inservice often to help them leam how to use technology to do 
research and create finished projects. We always make training sessions hands-on experiences as that is 
the only way to really leam how to use technology. 

At the beginning of this school year, we added an LCD projector and a 32" Destination Gateway 
computer to the library equipment so training could be seen on large screens. Because of the technology 
already in place in the library, we received a $15,000 grant from the state to create a class called 
Generation www. Y. The class was established in Washington state and other states were given the 
opportunity to try the class and use their curriculum. With the money we purchased 8 more computers, a 
scanner and a digital camera. The library now has 23 computers for research needs. I teach 2 Generation 
www. Y classes in the library. The students leam a variety of computer skills i.e. setting up computers, 
Windows 95, e-mail, the Internet, Power Point, Front Page, collaboration, digital images, etc. The 
students partners up with a teacher and worked with that teacher to create a technology infused lesson. 
All of the students showed a tremendous amount of growth in their technology skills and presentation 
abilities. The teachers enjoyed having students spend time teaching them technology and the teacher's 
classes enjoyed the technology driven lessons. Next year the students from this class will become 
technology proctors for different curriculum departments. They will work with all the teachers in that 
department to infuse technology into their classes. I will teach a new group of students next year who 
will replace the 1st group when they graduate. 

Next school year we will start the year with 12 more computers. That will bring us up to 35 computers. 
We aren't up to 60 yet, but this is only the end of year 2 in the 5 year plan. All other technology for the 
plan is in place at this time. The teachers have been inserviced as a whole staff, in curriculum 
departments, in small groups, and individually. Students are always inserviced on the best sources to 
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Ncjjth High School, KS ' 

meet their specific needs when they use the library as a class. They are also helped individually to find 
information to meet their learning objectives. Training will never be completed. There will always be 
newer and better sources to introduce the students and staff to. It is a challenge, but it also makes every 
day exciting. 

-###- 



r South Harrison Community Schoolsl 
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Alaska 

Jerry Schoenberger 
Principal 

Juneau School District 
10014 Crazy Horse Pike 
Juneau, AK 99801 
Phone: (907) 463-1700 
Email: schoenbj@jsd.kl2.ak.us 

Alabama 

Sammy Clanton 
Superintendent 

Cherokee County School District 
130 East Main St. 

Centre, AL 35960 
Phone: 256-927-5502 
Email: 

Arizona 

Kathy Tucker 
Curriculum Innovation Specialist-Technology Literacy 
Kayenta Unified School District 
PO Box 337 
Kayenta, AZ 86033 
Phone: (520) 697-2049 
Email: ktucker@kayenta.kl2.az.us 



Rick Clifton 
Teacher/Technology Coordinator 
Cherokee Co. School District 
130 East Main St. . 

Centre, AL 35960 
Phone: (256) 927-5502 
Email: 



Janet Valentour 
Middle School Teacher 
Juneau School District 
10014 Crazy Horse Pike 
Juneau, AK 99801 
Phone: (907) 463-1899 
Email: valentoj@jsd.kl2.ak.us 
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Spotlight School Contacts 




California 

Cathy Bean 
Superintendent 
Gold Oak Union 
3171 Pleasant Valley Rd. 

Placerville, CA 95667-9297 

Phone: (530) 626-3150 

Email: Ccbean@goldoak.edcoe.kl2.ca.us 

Colorado 

Tom Reed 

Teaching, Learning, & Tech Specialist 
Cherry Creek School District 
14188 E. Briarwood Avenue 
Englewood, CO 801 12 
Phone: (303) 486-4000 
Email: 

Linda Maccagnan 

Teaching, Learning, & Tech Specialist 
Cherry Creek School District 
14188 E. Briarwood Ave. 

Englewood, CO 801 12 
Phone: (303) 486-4000 
Email: 

Conneticut 

Bill Derry 

Supervisor of Library Media 

New Haven Public Schools 

54 Meadow St. Gateway Center 

New Haven, CT 06519 

Phone: (203) 946-6363 

Email: bill.derry@new-haven.kl2.ct.us 





John Umekubo 
Director of Technology 
Lennox School District 
10319 Firmona Avenue. 
Lennox, CA 90304 
Phone: (310) 330-4950 
Email: john@lennox.kl2.ca.us 



Steve Cowdrey 
Director of Technology 
Cherry Creek School District 
14188 E. Briarwood Ave. 
Englewood, CO 80112 
Phone: (303) 486-4000 
Email: 



Jean Lowery 

New Haven Public Schools 
New Haven, CT 
Phone: 

Email: bishop.wds.school@snet.net 



Florida 
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Spotlight School Contacts 

Toby Ford 

Mathematics Teacher 

Okaloosa County School District 

c/o Diane Holman - 120 Lowery Place, SE 

Fort Walton Beach, FL 32548 

Phone: (850) 833-6374 

Email: bear@cyou.com 
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Diane Holman 

District Technology Specialist 
Okaloosa County School District 
120 Lowery Place, SE 
Fort Walton Beach, FL 32548 
Phone: (850) 833-6368 
Email: 



Georgia 

Beth Carlisle 
3rd Grade Teacher 

Edgewood Elementary/Muscogee Co. 
3835 Forrest Rd. 

Columbus, GA 3 1 907 
Phone: (706) 569-2510 
Email: 



Jo Flynn 

Kindergarten Teacher 

Edgewood Elementary/Muscogee Co. 

3835 Forrest Rd. 

Columbus, GA 3 1907 
Phone: (706) 569-2510 
Email: 



"TOP^ 



Iowa 

Joanne Dorhout 

District Technology Coordinator 
Waterloo Community School 
1516 Washington St 
Waterloo, I A 50702 
Phone: (319) 291-4816or 4800vm 
Email: jod@waterloo.kl2.ia.us 

Idaho 

Betsy Goeltz 
Technology Director 
Blackfoot School District 
270 E. Bridge St. 

Blackfoot, ID 83221 
Phone: (208) 782-9548 
Email: goelb@d55.kl2.id.us 

IUinios 



Vaughn Murphy 

Instructional Technology Consultant 
Hearland AEA II 
6500 Corporate Dr. 

Johnston, I A 50131 

Phone: (515) 270-9030 

Email: vmurphy@aeall.kl2.ia.us 



Deb McGrath 
Technology Director 
Kuna School District 
610 N. School Ave. 
Kuna, ID 83634 
Phone: (208) 922-1005 
Email: 
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Spotlight School Contacts 

Shirley Berry 
Director 

Chicago Public Schools 

Medill Prof. Tmg. Ctr., 1326 West 14th PI. 

Chicago, IL 60608 

Phone: (773) 553-6260 

Email: sberry@csc.cps.kl2.il.us . 



Jeri Kinser 

Instructional Technology Specialist 

Illinois Migrant Council 

28 E. Jackson Blvd., Suite 1600 

Chicago, IL 60604 

Phone: (312) 663-1522 

Email: jeri.kinser@estrella.org 

Indiana 

Linda Burnham 

Director of Technology 

South Harrison Ctornmunity School Corp. 

315 South Harrison Dr. 

Corydon, IN 47112 

Phone: (812) 738-2168 

Email: bumham@corydon.shcsc.kl2.in.us 
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Minerva Garcia-Sanchez 

Projects Administrator 

Chicago Public Schools 

1326 West 14th Place, Rm. 104 

Chicago, IL 60608 

Phone: (773) 553-6260 

Email: msanchez@csc.cps.kl2.il.us 



Barb Park 



Hoopsteston Area School District - Migrant Project 
615 E. Orange St. 

Hoopeston, IL 60942 

Phone: (217) 283-5411- 217-283-6665 

Email: parkba@hoopeston.kl2.il.us 



Angela Miller 
Teacher 

South Harrison Community School Corp. 
315 South Harrison Dr. 

Corydon, IN 471 12 

Phone: (812) 738-2168 

Email: millera@corydon.shcsc.kl2.in.us 



Kansas 

Shawn Morris 

Teacher/ Media Specialist 

Wichita North High School/ 

Wichita USD #259 

1437 Rochester 

Wichita, KS 67203-2967 

Phone: (316) 973-6175 H#316 744-8660 

Email: nolibl@feist.com 



Howard Pitler 
Principal 

L'Ouverture Computer Technology Magnet/Wichita 

USD#259 

1539 Ohio 

Wichita, KS 67214-1526 
Phone: (316) 973-5050 
Email: hpitler@brookmms.com 



Kentucky 
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Spotlight School Contacts 

Karen Cheser 

District Instructional Supervisor 
Kenton County Schools 
20 Kenton Lands Road 
Erlanger, KY 41018 
Phone: (606) 344-8888 
Email: kcheser@kenton.kl2.ky.us 

LESS 

Louisiana 

Sylvia Bienvenu 

Evaluation Consultant, State Dept, of Ed 

State Dept, of Education 

805 Rue Royale 

New Iberia, LA 70563 

Phone: (318) 367-9613 

Email: sylb@aisp.net 
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Vicki Fields 

District Technology Coordinator 

Kenton County 

20 Kenton Lands 

Erlanger, KY 41018 

Phone: (606) 344-8888 

Email: vftelds@kenton.kl2.ky.us 



Bill Morrison 

Director, Educational Technology 

Rapides 

P.O. Box 1230 

Alexandria, LA 71309-1230 

Phone: (318)449-3154 

Email: morrison@rapides.kl2.1a.us 






Ann Grady 

Director, Boston Public Schools 
Boston Public Schools 

Office of Instructional Technology 55 Malcolm X Blvd., Bldg. 

#1, Cluster #13 

Roxbury, MA 02120 

Phone: (617) 635-8880 

Email: a grady@boston.kl2.ma.us 



Massachusets 

Mary Benson 

Technology Project Manager 
Boston Public Schools 
Boston, MA 

Phone: (617) 635-8880, 406 

Email : mbenson@oit .boston. k 1 2 .ma.us 



Maryland 

Elizabeth Glowa 

Director, Instructional Technology 
Montgomery County Public Schools 
CESC, 850 Hungerford Dr., Rm 55 
Rockville, MD 20850 
Phone: (301) 279-3564 
Email: liz_glowa@2fc.mcps.kl2.md.us 



Marlene Hartzman 

Director, Educational Accountability 

Montgomery County Public Schools 

850 Hungerford Dr. Rm 1 1 

Rockville, MD 20850 

Phone: (301) 279-3448 

Email: 
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Spotlight School Contacts 

Nancy Kepf 
Research Analyst 

Montgomery County Public Schools 
850 Hungerford Dr. 

Rockville, MD 20850 

Phone: (301)279-3186 

Email: nancy Jkepf@fc.mcps.k 1 2 .md.us 
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Barbra Reeves 

Director of Instructional Technology 
Maryland State Department of Education 
200 W. Baltimore St. 

Baltimore, MD 21201 
Phone: 410-767-0382 
Email: 

Reeves_Barbara / msde@netman.msde.state.md. 



Maine 

Robert Emrich 

Technology Coordinator/ Instructor 

Maine SAD #4 

7 Campus Dr 

Guilford, ME 4443 

Phone: (207) 876-4625 

Email: 

Michigan 

Joan Buffone 

Asst. Superintendent of Instruction 
East Detroit Public Schools 
151 15 Deerfield St. 

Eastpointe, MI 48021 
Phone: (810) 445-4425 
Email: 



William Thompson 
Technology Coordinator 
Maine SAD #4 
7 Campus Dr. 

Guilford, ME 4443 
Phone: (207) 876-4625 
Email: 



Lisa M. Cardamone 
Technical Resource Facilitator 
East Detroit Public Schools 
15115 Deerfield St. 

Eastpointe, MI 4802 1 
Phone: (810) 445-4425 
Email: lclmeds@moa.net 



Terri Spencer 

Director of Educational Technology 
East Detroit Public Schools 
151 15 Deerfield St. 

Eastpointe, MI 48021 
Phone: (810) 445-4402 
Email: tmspencer@aol.com 



Minnesota 

Julie Beddow-Schubert 
Technology Integrationist 
Little Falls Community Schools 
16591 Riverwood 
Little Falls, MN 56345 
Phone: (320) 616-5208 
Email: 



Gregg Pearce 
Technology Integrationist 
Little Falls Community Schools 
300 6th St. SW 
Little Falls, MN 56345 
Phone: (320) 616-6208 
Email: gpearce@lfalls.kl2.mn.us 
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Spotlight School Contacts 

Missouri 

Stephanie Moore 
Teacher 

Jennings School District 
8888 Clifton Ave. 

Jennings, MO 63136 

Phone: (314) 867-8900 

Email: hot005@mail.connect.more.net . 

Diane Vaughan 
Evaluator/ MINTS Project 
Jennings School District 
8888 Clifton Ave. 

Jennings, MO 63136 
Phone: 314-867-8900 
Email: 

Mississippi 

Cameron Abel 
Teacher 

Drew School District 
286 West Park Ave. 

Drew, MS 38737 

Phone:(662)745-6657 

Email: cameron@microsped.com 

Montana 

Barbara Ridgway 

District Technology Coordinator 

Helena School District No. 1 

55 South Rodney 

Helena, MT 59601 

Phone: (406)447-8526 

Email: bridgway 




North Carolina 



00001)0001551 



Terry Stewart 
Superintendent 
Jennings School District 
8888 Clifton Ave. 
Jennings, MO 63136 
Phone: (314) 867-8900 
Email: 



Dennis Silas 
Technology Coordinator 
Drew School District 
286 West Park Ave. 

Drew, MS 38737 

Phone: (662) 745-6657 

Email: dennis.silas@mailcity.com 



Sandra Smith 

K-5 Technology Teacher/ Tech. Ctr. Coordinator 

Helena School District No. 1 

1600 8th Avenue 

Helena, MT 59601 

Phone: (406) 447-8943 

Email: ssmith@helena.kl2.mt.us 
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Spotlight School Contacts 

Mary Forrest 

Director of Media & Technology 
Carteret County Schools 
PO Drawer 600 - 107 Safrit Dr. 

Beaufort, NC 28516 
Phone: (252) 728-4583 
Email: 

North Dakota 

Sam Azure 

Principal, Theodore Jamerson Elementary 
Theodore Jamerson Elementary School 
3315 University Dr. 

Bismarck, ND 58504 
Phone: (701) 255-3285 X. 305 
Email: 

Nebraska 

Dick Albrecht 
History/ French Teacher 
Ainsworth Community Schools 
520 East 2nd St 
Ainsworth, NE 69210 
Phone: (402) 387-1209 
Email: 

New Hampshire 

Ethel Gaides 
Technology Coordinator 
Campton School District, SAU #48 
Rt. #175 

Campton, NH 03223 
Phone: (603) 726-3931 
Email: egaides@hotmail.com 

New Jersey 
Mary Rau 

English/ Speech Teacher 
Ainsworth Community Schools 
520 East 2nd St 
Ainsworth, NE 69210 
Phone: (402) 387-1983 
Email: 



. 000000001552 

Ralph Lewis 

Director of Student Services and Testing 

Carteret County Schools 

PO Drawer 600 - 107 Safrit Dr. 

Beaufort, NC 28516 

Phone: (252) 728-4583 

Email: 



Rick Peek 

Technology Coordinator 

Theodore Jamerson Elementary School 

3315 University Dr. 

Bismarck, ND 58504 

Phone: (701) 255-3285 X. 305 

Email: 



Nicki Vanek 

Classroom Teacher 

SAU #48 Campton School District 

Rt. #175 

Campton, NH 03223 

Phone: (603) 726-3931 

Email: nickivaneck@hotmail.com 



Paul Cohen 

Supervisor of Social Studies/ Media K-12 
East Brunswick Public Schools 
760 Route 18 

East Bmnswick, NL08816 
Phone: (732) 613-6766 
Email: coheup51a@ebnet.org 
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Spotlight School Contacts 

Evelyn Ogden 

Deputy Superintendent 

East Brunswick Public Schools 

760 Route 18 

East Bnmswick, NJ 08816 

Phone: (732) 613-6760 

Email: ogdene51a@ebnet.org 

New Mexico 
Phyllis Ivy 

District Technology Coordinator 
Roy Municipal Schools 
Drawer 430 
Roy, NM 87743 
Phone: (505) 485-2276 
Email: aivey@etsc.net 
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Karen Warner 

East Brunswick Public Schools 
760 Route 1 8 

East Brunswick, NJ 08816 

Phone: (732) 613-6760 

Email: karen_wamer@rocketmail.com 



William Ortiz 

Federal Projects Coordinator 

Santa Rosa Consolidated Schools 

HCR 68, Box 9 

New Kirk, NM 87443 

Phone: (505) 472-3633 or 505-799-8108 

Email: 



Nevada 

Jan Chism Julie Doyle 

Technology Coordinator Library /Media specialist 

Clark County School District, Lummis Elementary Clark County School District 
9000 Hillpointe 9000 Hillpointe 

Las Vegas, NV 89134 Las Vegas, NV 89134 

Phone: (702) 799-4380 Phone: (702) 799-4380 

Email: jan-e.-chism@aspen.interact.kl2.nv.us Email: julied@interact.ccsd.net 

New York 

Kenneth Eastwood 
Asst. Superintendent 
Oswego City School District 
120 East First St. 

Oswego, NY 13126 
Phone: 3 15-^4 1-5838 
Email: 




Ohio 
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Spotlight School Contacts 

Lynda Gallagher 
Language Arts Teacher 

Middletown City Schools - Vail Middle School 
1415 Girard Ave. 

Middletown, OH 45044 
Phone: (513) 420-4528 
Email: 

Oklahoma 

Anne Masters 

Director, Media Services & Instructional Tech 

Norman Public Schools 

4100 N. Flood 

Norman, OK 73069 

Phone: (405) 366-5863 

Email: 

Nancy O'Brien 
Superintendent 
Norman Public Schools 
131 S. Flood 
Norman, OK 73069 
Phone: (405) 366-5955 
Email: obrienm@erols.com 




Pennsylvania 

Judy Yoho 

Technology Coordinator 
Keystone Central 
95 W. Fourth Street 
Lock Haven, PA 
Phone: (717) 893-4900 
Email: 
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Dona Gardner 

Principal, Vail Middle School 

Middletown City Schools - Vail Middle School 

1415 Girard Ave. 

Middletown, OH 45044 
Phone: (513) 420-4528 
Email: 



Vicki Medlin 
4th/5th Grade Teacher 
Norman Public Schools 
432 South Lahoma 
Norman, OK 73069 
Phone: (405) 321-1646 
Email: 
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Spotlight School Contacts 

Holly Barton 
Library Media Specialist 
Hope Valley Elementary School 
Main St. 

Hope Valley, RI 2832 
Phone: (401) 539-2321 
Email: bartonh@ride.ri.net 

South Carolina 

Andrea Daniels 

Parent Technology Outreach Consultant 

Richland County School District One 

1225 Oak Street, Rm 201 A 

Columbia, SC 29204 

Phone: (803) 733-3004 

Emai 1 : adaniels@richlandone.org 
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Diane Morris 

Asst. Tech/ Middle School Tech Facilitator 
Bristol Warren Regional School District 
Child St. 

Warren, RI 01885 
Phone: (401)245-2010 
Email: dmm@etal.uri.edu 



Jim Hockman 

Richland School District 
4100 Claremont Dr. 

Columbia, SC 29205 
Phone: 

Email: jhockman@richlandone.org 



Ida Thompson 

Richland School District 
2 Torrington Ct. 

Columbia, SC 29222 
Phone: 

Email: irwt@richlandone.org 

South Dakota 

Kelly Lane 

Middle School Teacher 
Douglas Public School 
401 Tower Rd. 

Box Elder, SD 57719 
Phone: (605) 923-4380 
Email: 



T 




Lennie Symes 
Director of Technology 
Huron Public School District 
1878 McClellan 
Huron, SD 57370 
Phone: (605) 353-6992 
Email: 



Tennessee 
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Spotlight School Contacts 

W. Perry Brown 
• 

Anderson County Schools 
2519 Federal Drive 
Talbott, TN 37877 
Phone: (423) 475-9446 
Email: 

Texas 

Ismael Salas 

Career and Technology Teacher 
Fabens Independent School District 
PO Box 697, 821 N.E. "G" Ave. 
Fabens, TX 79838-0697 
Phone: (915) 764-2498 
Email: isalas@fabens.kl2.tx.us 
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Johanna Cole Whitley 

Anderson County Schools 
514 Mariner's Point 
Clinton, TN 37716 
Phone: (423) 457-8796 
Email: 



Martha Veale 

Technology Coordinator 

Fabens Independent School District 

PO Box 697, 821 N.E. "G" Ave. 

Fabens, TX 79838-0697 

Phone: (915) 764-2498 

Email: mveale@fabens.kl2.tx.us 



Utah 

Vickie Guvmon 

Teacher, Technology Specialist 

Sally Mauro Elem./Carbon 

20-2nd Avenue 

Helper, UT 84526 

Phone: (435) 472-5311 

Email: vickie.guymon@svc.sesckl2.ut.us 



Rob Henderson 
Technology Technician 
BIA, Aneth Community School 
PO Box 600 

Montezuma Creek, UT 84534 
Phone: (435) 651-3271 X701 
Email: rob@aneth.sanjuan.kl 2.ut.us 



Bryan Snow 
Technology Director 
BIA, Aneth Community Schools 
PO Box 600 

Montezuma Creek, UT 84534 
Phone: (435) 651-3271, X700 
Email: bsnow@aneth.sanjuan.kl 2.ut.us 




Virginia 
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Spotlight School Contacts 

Maureen Boland 



Linda Gill 



Second Grade Teacher, Mantua Elementary 
Mantua Elemntary School 
3008 Woodlawn Ave. 

Falls Church, VA 22042 

Phone: (703) 241-2194 

Email: mboland@fc.fcps.kl2.va.us 

Sarah Skerker 
Communications Teacher 
Mantua Elementary/Fairfax County Public 
9107 Homer Ct. 

Fairfax, VA 2203 1 

Phone: (703) 645-6300 

Email: sskerker@fc.fcps.kl 2. va.us 

Vermont 

Gregg Martin 

Director of Information Tech. Services 
Addison Central Supervisory Union 
Charles Avenue 
Middleburg, VT 05753 
Phone: (802) 382-1277 
Email: gmartin@acsu.kl2.vt.us 



Director, Technology and Information Services 

Henrico County Schools 

P.O. Box 23120 

Richmond, VA 23223-0420 

Phone: (804) 652-3755 

Email: lagill@henrico.kl2.va.us 

Charles Stallard 

Director, Technology and Information Services 
Henrico County Public Schools 
PO Box 23120, 3820 Nine Mile Rd. 

Richmond, VA 23223-0420 
Phone: 804-652-3755 
Email: 



Jan Willey 

Asst. Superintendent 

Addison Central Supervisory Union 

Charles Avenue 

Middleburg, VT 05753 

Phone: (802) 382-1277 

Email: jwilley@acsu.kl2.vt.us 



Schools 




Washington 
Doug Hamilton 

Manager, Instructional Technology 
Evergreen School District 
13501 NE 28th St., PO Box 8910 
Vancouver, WA 98668-8910 
Phone: (360) 604-4085 
Email: 

Wisconsin 
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Spotlight School Contacts 



Sherlyn Brown 
Teacher 

Burroughs Middle School/Milwaukee Public 
Schools 

6700 N. 80th St. 

Milwaukee, WI 53223 
Phone: (414) 353-3220 
Email: 

West Virginia 

Rachelle Harter 
Teacher 

Hundred High School 
P.O. Box 830 



000000001558 

Kathy Swope 

Performance Assessment Specialist 
Milwaukee Public Schools 
PO Box 2181 

Milwaukee, WI 53201-2181 

Phone: (414) 475-8303 

Email: swopekr@maii.milwaukee.kl2.wi.us 



Jim McGlumphy 
Technology Coordinator 
Ohio County Schools 
2203 National Rd. 



Hundred, WV 26575 
Phone: 304-455-2441 
Email: 

Bernard Shackleford 
Teacher 

Hundred High School 
P.O. Box 830 
Hundred, WV 26575 
Phone: (304) 775-5221 
Email: 

Wyoming 

Kimball Croft 
Technology Director 
WCSD #1 
1800 Howell 
Worland, WY 82401 
Phone: (307) 347-3412 
Email: 



Wheeling, WV 26003 
Phone: (304) 243-0448 
Email: 



Conference Agenda | Spotlight Schools | Spotlight School Contacts | Conference White Papers | Featured Speakers | Acknowledgements 






Return to Conference Main Page 



EIT* 

Page last updated on July 20, 1999 by [eid] 



ill 

vww.ed.gov/Technology/TechConf/1999/sptlitcontcts.html (14 of 14) [4/24/2001 2:32:28 PM] 

ERJC 




WMJp Papws, Table of Contents 




JQM3QI0M . 

The Secretary's Conference on Educational 
Technology 

Conference Research 
Materials 

1. "Perspectives on Technology and Education Research: Lessons 
from the Past and Present" 

Margaret Honey and Katherine McMillan Culp, EDC/Center for 
Children and Technology, EDC/Center for Children and 
Technology, and Fred Carrigg, Union City New Jersey Board of 
Education, 1999. 

2. "Convergent Analysis: A Method for Extracting the Value from 
Research Studies on Technology in Education." 

Cathleen Norris and Jennifer Smalka, College of Education, 
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Perspectives on Technology and 
Education Research: Lessons from the 

Past and Present 1 

Margaret Honey, EDC/Center for Children and Technology i 
Katherine McMillan Culp, EDC/Center for Children and Technology 
Fred Carrigg, Union City New Jersey Board of Education 

This paper offers a perspective that grows out of what we, at EDC's Center for Children and Technology, 
have learned from nearly three decades of research on educational technology . Rather than providing a 
detailed account of what we now know about the impact of technology on learning, we discuss where the 
research field is heading and review what we think of as the most promising directions for technology's 
role in education (President's Committee of Advisors on Science and Technology, 1997; Bransford, 
Brown, Cocking, 1999; Coley, Cradler, & Engel, 1997). 

Several factors are prompting us to think differently about research. Each one is based in our 
observations of steady growth and change on other fronts. 

• First, there are changes in the nature of the technological elements involved in the research; 

• Second, there are changes in the kinds of research questions being asked. 

• And third, there are changes in how research is being done and the methods being used. 

Technological change. Throughout the 1970s and '80s, technical innovation brought increasingly 
diverse and more powerful technological tools into schools. Early studies sought to demonstrate the 
impact of technologies or software on student learning and they were tied very specifically to the 
particular technologies used by the subjects of the study. These technologies were typically text-based, 
locally networked or stand-alone computer-assisted instruction applications. As these technologies have 
become outdated and replaced by graphics-rich and networked environments, the studies that looked at 
impact on students have been outdated themselves. Additionally, because these studies tended to focus so 
specifically on particular technologies and their impact, they contributed little to the larger and more 
challenging project of learning about the generalizable roles that technologies can play in addressing the 
key challenges of teaching and learning, and about optimal designs for such technologies. The pace of 
both technological development and the introduction of new technologies into educational settings has 
dramatically accelerated during the past decade. The combination of computation, connectivity, visual 
and multimedia capacities, miniaturization, and speed have radically changed the potential for 
technologies in schooling. These developments are now making it possible for technologies to be 
designed and deployed to produce powerful and linked technologies that can substantially address some 
of the core problems of education. For example, there are currently existing technologies that are ideally 
suited to ameliorating problems like teacher isolation, access to rich and substantial resources, and parent 
involvement in schooling (Glennan, 1998; Hawkins, 1996; Koschmann, 1996; Pea, Tinker, Linn, Means, 
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Bransford, Roschelle, Hsi, Brophy, & Songer, in press). 
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Changes in the questions being asked. As the technologies themselves have changed, so have our 
research questions. We began, in the 1970s, by asking questions about whether certain kinds of 
computer-based activities improved student learning. Studies did find improvements in student scores on 
tests closely related to material covered in computer-assisted instructional packages (Kulik & Kulik, 
1991). But these studies did nothing to help us understand how technologies might, or might not, help to 
support the kinds of sustained and substantial inquiry and analysis that we all want our children to 
achieve. Of course these studies did not help us learn about these more complicated issues because they 
did not ask the more complicated questions that would be required in order to begin to learn about these 
issues. More specifically, a key problem with these studies is that they did not acknowledge that effective 
technology use needs to be embedded in a larger process of school change— that understanding the impact 
of technology integration requires understanding technology use in a social context— and instead tended 
to treat technology as a discrete and isolated, yet, it was hoped, overwhelmingly powerful input. 



Implicit in these initial strands of research was an assumption that schooling is a "black box" (Tally, 
1998). Research attempting to answer the question "Does technology improve student learning?" had to 
eliminate from consideration everything other than the computer itself and the evidence of student 
learning. Teacher practices, student experiences, pedagogical contexts, and even what was actually being 
done with the computers - all of these factors were bracketed out. This was done so that the researcher 
could make powerful, definitive statements about effects - statements unqualified by all the complicated, 
gritty details of actual schooling. 

The problem was that all the studies conducted in this way - and there were hundreds - told educators 
clearly that specific kinds of technology applications, such as integrated learning systems, could improve 
students' scores on tests of discrete information and skills, such as spelling, basic mathematics, 
geographical place names, and so on. But these studies were not able to tell educators very much that 
helped them address the larger challenge of using technology to support students in developing capacities 
to think creatively and critically, and to learn to use their minds well and deeply. 

It has become clear through past research on the impact of technology on education that technologies by 
themselves have little scaleable or sustained impact on learning in schools. In order to be effective, 
innovative and robust technological resources must be used to support systematic changes in educational 
environments that take into account simultaneous changes in administrative procedures, curriculum, time 
and space constraints, school-community relationships, and a range of other logistical and social factors 
(Chang, Honey, Light, Moeller & Ross, 1997; Fisher, Dwyer, & Yocam, 1996; Hawkins, Spielvogel & 
Panush, 1996; Means, 1994; Sabelli & Dede, 1998; Sandholtz, Ringstaff & Dwyer, 1997). 



While the pressure continues to develop answers about how technologies may contribute to student 
learning, there has been increasing recognition that technology is a crucial player in a more complex 
process of change that cannot be accomplished by technological fixes alone. As a result, researchers are 
increasingly asking questions about how technology is integrated into educational settings; how new 
electronic resources are interpreted and adapted by their users; how best to match technological 
capacities with students' learning needs; and how technological change can interact with and support 
changes in many other parts of the educational process, such as assessment, administration, 
communication, and curriculum development. 



Changes in methods. Answering these kinds of questions also requires the expansion or improvement of 
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a whole range of interconnected resources— including technologies, teachers, and social services— that 
cannot be isolated for study the way a single software program can be. Further, the kinds of outcomes 
associated with changing and improving the circumstances of teaching and learning are much more 
holistic than those measured by most standard assessment practices, and they require more sophisticated 
strategies of the researcher who is attempting to capture and analyze them. To explore how best to use 
technology in the service of these goals requires looking at technology use in context, and gaining an 
understanding of how technology use is mediated by factors such as the organization of the classroom, 
the pedagogical methods of the teacher, and the socio-cultural setting of the school. 

Researchers are now emphasizing questions about the intersections of design, learning, school culture 
and practices, and other factors that shape the impact technologies can have in schools. A key 
recommendation growing out of the President's Committee of Advisors on Science and Technology is 
the need for large-scale, longitudinal studies that examine the consequences of technology use in school 
settings in concert with a broad range of factors. 

To illustrate how technologies can be used to support and extend a broad-based program of education 
change, we would like to use the example of a comprehensive program of reform that has taken place in 
the Union City New Jersey schools. 

The Union City Story 

Union City, New Jersey, is located in Hudson County, directly across the Hudson River from Manhattan. 
With 60,000 residents in 1 .4 square miles, it is the most densely populated city in the United States. The 
predominant ethnic makeup of Union City is Cuban, though recent arrivals from the Caribbean, Central 
and South America, as well as long-time Italian residents, add to the diversity of the city's population. Of 
the 9,803 students in the District's eleven schools, 93% are Latino, 68% of whom do not speak English at 
home. Thirty two percent of the students are enrolled in the District's bilingual/ESL program. The 
Brookings Institute classified Union City as one of the 92 most impoverished communities in the United 
States; 27.5% of all children live below the poverty line and 84% receive free or reduced lunches. 

The Center for Children and Technology first began to work with the Union City schools in 1992. We 
were brought into the district by Bell Atlantic to assist with an initiative known locally as Project 
Explore. Back in 1992, Project Explore represented an innovative home-school networking initiative. It 
supplied 135 seventh-grade students and 20 teachers with networked computers at home and at school. 
While Project Explore has been the focus of our research, our work with Union City extends beyond this 
effort. In 1995, in collaboration with the Union City Board of Education and Bell Atlantic, we were 
awarded a grant from the National Science Foundation to conduct a project called Union City Online: An 
Architecture for Networking and Reform. This effort built upon the work of Project Explore, and 
extended the networking infrastructure to the District's remaining ten schools to help launch a number of 
other projects to help develop the human infrastructure - the people resources that it takes to make a 
complex project like this succeed and remain successful overtime. Another core goal of Union City 
Online was to take a substantial and sustained look at the relationship between networked technology and 
education reform (Honey, Carrigg Hawkins, 1998). 

What is critical in this story is understanding what has happened in Union City during the past 10 years . 
In 1989, the Union City schools failed in 44 out of the 52 categories that the State of New Jersey uses to 
determine the effectiveness of their school districts. They were failing in areas such as student 
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attendance, drop-out rates, and scores on standardized tests, and as a result they were facing state 
takeover. Like many urban districts, Union City was also facing many obstacles to correcting these 
deficiencies, including language barriers, parents with limited formal education, and students with little 
incentive to stay in school. 



Rather than lose local control of the school district, however, Union City decided to face these challenges 
head on and drastically reform the entire educational system. The District formulated and implemented a 
five-year Corrective Action Plan calling for systemic changes in the educational system. Using their own 
version of a whole-language approach to learning — which put literacy front and center in their reform 
- efforts — the District focused on creating a curriculum which would support students in moving away 
from rote learning and toward the development of thinking, reasoning and collaboration skills. In order to 
facilitate these goals, the district did a number of things, including the following: 

• Classes were extended in most subject areas to 1 1 1 -minute periods in the elementary and middle 
schools, and 80-minute periods in the high schools. 

• In-service training for teachers was increased from 8 hours a year to 40 hours. 

• Buildings were refurbished, windows were replaced, and classrooms and hallways were painted. 

• Individual student desks were replaced by cooperative learning tables. 

• Textbooks for individual students were replaced with class libraries. 

Union City chose to implement the reforms first in the elementary classrooms, then add classes year by 
year until reform reached every grade level. This decision meant that no student schooled in a reformed 
learning environment entered a new grade only to face the former method of instruction. Furthermore, 
the District did not have to face on an unmanageable scale the inevitable headaches that arise during 
renovations and the first years of new curricula. It also meant the District was able to take the lessons 
learned from each successive implementation and apply them toward easing the transition in subsequent 
years. 

In addition to curriculum reforms, substantial increases in the District's operating budget played a critical 
role in Union City's efforts. Over the past eight years, the budget for the Union City School District 
increased from $37.8 million in 1989 to $100 million in 1997 as a direct result of equitable school 
funding legislation, known in New Jersey as the Quality Education Act (QEA). 



Beginning in 1993, Union City also made a deliberate decision to invest substantially in technology 
resources. They did this largely out of equity considerations, believing that urban students would once 
again risk falling drastically behind suburban students if they did not have access to state-of-the-art 
technological resources. The District built fiber backbones in each of its eleven schools. Approximately 
85% of the 2,200 instructional computers - those in classrooms, media centers, and computer labs - are 
part of a district-wide network that connects the schools, two public libraries, city hall, and the local 
daycare center through T-l lines back to the central office servers. With a ratio of four students per 
computer, Union City is now one of the most, if not the most, wired urban school district in the United 
States. 



The Center for Children and Technology has been conducting research in relation to both the 
NSF-funded work in Union City and the Project Explore initiative. Our most recent examination of the 
impact of the district reforms and the impact of technology on student learning resulted in three 
important findings: 
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• The educational reforms have had a substantial impact on students' standardized-test performance, 
particularly at the K-8 level, where the reforms have been in place the longest. 

• The Explore students (those with home as well as school access to technology) gained a substantial 
"leg up" during the first year of the project, scoring significantly better than their district peers in 
writing and mathematics. This increase, however, is not due to technology alone, but to increased 
expectations and to the dedication of teachers and administrators in ensuring that this group of 
students would excel. 

• Writing is the one area where deep and sustained access to technology has made a difference. At 
the 7th-, 8th-, and 9th-grade levels. Explore students do significantly better than their non-Explore 
peers on the writing portion of state tests. 

Our research suggests that deep and sustained access to technology has the potential to have a positive 
impact on both students' learning and on the school community's views of their students' capabilities. But 
our research also suggests that technology in and of itself, in the absence of other components of school 
reform, would not produce these kinds of changes. We have identified eight key reform strategies 
integral to the Union City school district's success. These are: 

• Instructional leadership at the building level 

• Effective school improvement teams 

• Extensive professional development in whole-language teaching approaches and cooperative 
learning 

• A strong emphasis on student creativity and the expression of ideas in multiple formats 

• An emphasis on providing different points of entry into a task for children working at different 
ability levels 

• A de-emphasis on remediation and an emphasis on learning for all 

• Establishment of classroom libraries and media-rich classroom environments 

• Multi-text approach to learning that includes the integration of technology into instruction. 

Union City has taught us a great deal about how research can focus on improving circumstances of 
learning, and on determining how technology can help make that happen. This requires viewing 
technology not a solution in isolation, but as a key component in making it possible for schools to 
address core educational challenges. A consensus is emerging that the larger issue that needs to be 
addressed across a wide range of iterative, collaborative research projects is gaining an understanding of 
the qualities of successful technological imiovations as they unfold and begin to have an impact within 
local, district, regional, and national contexts. 

As researchers have come to focus on these issues, a number of common characteristics have emerged in 
the design and methods involved in this type of research. 



Key assumptions of this kind of research include: 

• Recognizing that technologies in and of themselves rarely bring about substantial change in 
teaching and learning. 

• Understanding that the impact of technology on specific aspects of teaching and learning can be 
usefully understood only in context. Technologies matter only when harnessed for particular ends 
within the social contexts of schools. We are not suggesting that this eliminates the need for 
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careful formative research with users in experimental or laboratory settings. But it does mean that 
the research agenda is not completed when a robust application has been developed for use in 
learning settings. It means that a key phase of research must involve looking at how new 
technological applications can be integrated into school contexts and how they fit into the complex 

. process of school change. 

Methodological features of this kind of research include the following: 

• It is largely process-oriented. The researchers' goal is to understand how innovation occurs in 
schools, not just what the outcomes correlated with the innovation are. 

• It is oriented toward change rather than doing better within the old framework. Tools and 
programs that are interesting to study are those that support or act as catalysts for change in the 
organization of teaching and learning. 

• Teachers and researchers play an active role in interpreting technologies as tools for reforming 
schools and in supporting and sometimes guiding the change process. 

• It is multidisciplinary, combining elements of different fields, including: 

o anthropological lenses on the culture of schools and classrooms and kids' lives inside and 
outside them 

o developmental and cognitive psychology lenses on learning 
o sociological lenses on school institutions and school change. 

There are also important design elements that this type of research entails: 

• Long-term collaborations with educators. Teachers must be partners and co-constructors of the 
innovations and of the research process, rather than being viewed as subjects or passive recipients 
of the innovation. 

• Systemic integration and research on the impact of innovations across multiple levels of the school 
system. Isolated classroom experiments are being replaced by broad examinations of the roles 
technological innovations can play in the whole system of schooling, at the classroom, individual 
school, district, state, and national levels. This type of research includes "test-bed" studies that 
track long-term school changes that are technology-enhanced. 

What we have learned so far? Several broadly supported conclusions have emerged from this type of 

research. 

• We have begun to learn about the roles that specific technologies can play in helping to reorganize 
the education workplace. 

• We have become accustomed to defining our strategies and research questions from the point of 
view of education problems or challenges, rather than beginning from the technologies' 
capabilities. 

• We have come to appreciate the powerful role technology can play in creating new links between 
schools and the world outside the schools, connecting individuals, providing resources, and 
broadening the cultural and political contexts available to students and teachers for exploration and 
examination. 

• Most importantly, we have learned that research that is focused on change cannot be done at a 
distance, nor can it proceed from the assumption that the answers lie outside of the school 
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community. 



Our work in Union City has taught us a great deal about the value of working in collaboration where all 
parties are learning together and privileging the knowledge, expertise, and limitations that everyone 
brings to the task at hand. 
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The costs for implementing technology projects in K-12 classrooms - from the use of word processors in 
writing classes to visualization software in science classes - are significant. While the price of hardware 
has plummeted, the companion costs of administrator, teacher, and student time remain stubbornly high. 
Help is potentially available, however: literally hundreds of research reports on the use of technology in 
education have been published over the past 30 years. In those research reports are the successes and 
failures that we can learn from; the reports provide a window on others trials in bringing technology to 
the classroom. 

But frankly, there has been little impact of that research community's findings on the practitioner's 
community of the classroom. Here are several reasons for this breakdown: 

• Research articles are written, by and large, for other researchers; their style of reporting does not 
address "what can a teacher learn from this study that is applicable to their classroom, today." 

• There are many controlled studies that simply report, in a horserace format, who won and who 
lost, e.g., significantly more students wrote higher quality reports using a word process than those 
not using a word processor. These studies do not go on to provide an analysis of why - why did 
that outcome occur? 
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• Literature review articles tend to follow the model of the controlled studies and similarly 

summarize the research in terms of who won and who lost, e.g., "these n studies found that there 
was an impact of word processors on writing quality, while these m studies found no such impact." 
Or they make the following sort of report: "based on a review of the literature, these n factors are 
involved in the successful implementation of technology in the classroom." The list is a 
common-sense list of all the issues involved in making technology a success; again, the underlying 
"why" is not addressed. 



• There are a number of published articles that summarize the literature and attempt to tell the 
practitioner "what works." However, these articles tend to be at a very high level, e.g., use 
simulations, use multiple representations, etc. While good advice, these observations are quite 
general; the conditionality that should temper the application of these pieces of wisdom is typically 
not provided. 



The objective of this short paper, then, is present a method f< extracting value from the research 
literature that should benefit educational practitioners. We call this method, Convergent Analysis (CA). 
CA comprises a number of steps. 



First, we need to pose a question whose answer can benefit educational practitioners. Thus, rather than 
going to the literature to ask a broad question such as "does technology lead to increased student 
achievement," we ask a more focused, practitioner-oriented question: 

• under what conditions do computers lead to increased student achievement? 



That is, what are the issues that need to be addressed, what are the key factors and their values that are 
involved in leading to a positive learning outcome with technology? For example, from reading the 
research literature "time-on-task" can be seen as a key factor; for computers to have a positive impact on 
writing, children need to spend an "adequate" amount of time actually writing on computers. In looking 
at the range of research papers one can come to an understanding of the different tradeoffs that could 
result in providing children with "adequate time." 

In phrasing the question of computer impact in terms of "conditions under which. . ." an important 
opportunity has been created. After identifying those conditions for success the next step is identifying 
concrete actions that teachers, students, administrators, and parents can take towards realizing those 
conditions. For example, administrators can work towards getting funds to buy computers so that 
students can have adequate time writing on the computer. Curriculum coordinators can work towards 
organizing a curriculum unit to enable teachers to create lesson plans for the unit that enable students to 
have adequate writing time on computers. 

Second, we need to review the empirical studies in the literature and put them into a standardized format. 
And third, we need to look across all the studies in that standardized format, so as to compare and 
contrast the issues in each study. Clearly, CA is a time-consuming, detail oriented process! 

In what follows, then, we highlight the various steps of the CA process and provide examples of the 
nuggets of wisdom that we have extracted from the literature on writing education and technology using 
the CA method. 



Extracting Value from the Research Literature: A Multi-Step Process 
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Two major problems confronted us when we started reading research articles about the use of technology 
in writing education. 

1. Comparison across studies was not obvious: The research literature we found was exceedingly 
diverse in its reporting form and content. How did the issues and findings in one article relate to 
those in others? 



2. Findings were not focused on practitioner issues: The agendas of the researchers carrying out the 
studies were not necessarily the same as the agendas of the classroom practitioners. For example, 
we saw researchers structuring their report to highlight one issue: did the technology lead to a 
positive impact or not. In contrast, a teacher is more interested in the conditions that lead to the 
outcome, so they could know how to implement and adapt the technology in their classroom. 



In what follows, then, we describe how the Convergent Analysis method attempts to address these two 
problems. 



Step 1: Profiles: Standardizing Research Studies 

While some fields have de facto standards for research reporting (e.g., studies presented in major 
psychology or medical journals), the "field" of education and technology is not nearly as organized, and 
thus the format for reporting an empirical study varies widely. This diversity - a euphemism if ever there 
was one - makes accumulating the findings across studies exceedingly difficult. The meta-analytic 
method steers one course through this maze: only studies that admit of specific statistical characteristics 
are usable in the comparison. Unfortunately, many if not most of the research studies can't meet the stiff 
meta-analytic demands and are thus excluded from a literature review. But, just because a study isn't 
tightly quantitative doesn't mean that it is a bad study. We thus wanted to develop a method to analyze 
the literature that was more inclusive, and viewed the breadth in the research base as a feature, not a bug. 

Still further, we observed in reading paper after paper that researchers wrote for fellow researchers. The 
style of the research reports was clearly academically-oriented, and the content focused on issues of 
concern to researchers. For example, while a paper might contain an extended discussion of the 
theoretical framework for the study, it would say precious little about the details of actually running the 
study in a classroom setting. It is no wonder, then, that the research literature is not consulted by 
practitioners - researchers don't consider them their audience. 

To address both issues we developed a "Research Profile, " a template which now has about 75 
categories (e.g., enabling conditions such as teacher experience, technology availability and enactment 
conditions such as time on task, nature of the task, etc.) The Profile identified the issues that practitioners 
were concerned with. We consulted with education professionals in order to hone in on the categories of 
information relevant to classroom teachers as well as school administrators. Over 6 months of reading, 
rereading, and rerereading the research literature, we went through four major iterations of the profile. 
And, we still continue to tweak it! 

Now, filling out a profile for a research article is no mean feat! It takes hours and multiple readings of the 
paper in order to accurately fill in the cells of a profile. Interestingly, we reread papers we had originally 
reviewed before the profile was developed and we oftentimes changed our opinion and our understanding 
of the research study. The profile helped us focus on the truly salient issues in the research study. In 
effect, the labeled cells in the profile served as prompts to the reviewer; the profile scaffolded reviewers 
in getting at all the issues of a study. 
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Inasmuch as the articles in the literature that we profiled were not directed towards practitioners, it is not 
surprising that even after multiple readings we were not able to fill in many of the cells in a profile. 
Researchers did not include in their published articles information that was important for teachers who 
would want to either replicate a study or adapt the study to the particulars of their classrooms. 



Profiling 60 empirical research studies on writing education and technology is a major undertaking. We 
were fortunate, therefore, to enlist the aid of a graduate class of students at the University of North Texas 
in the College of Education. Over a 2 month period, 23 students working in teams created the online 
database of profiles available for public perusal. The findings described in the remainder of this article 
are based on our readings of these profiles. 



Step 2: Convergent Analysis Comparing Across Studies 

Once the literature has been put in a standardized format, it is possible to systematically examine the 
studies to identify patterns. We have called this focusing in, this triangulating process, "convergent 
analysis" (CA). For example, in looking across all the studies on writing education and technology, we 
first put those that showed children gaining benefit from using a word processor in one pile, and those 
that did not show benefit in another. Then, we asked, can we explain why those that did not seem to 
show benefit on the basis of other findings? In comparing across cells such as task, we saw the 
following: 

• Using a word processor changed the writing task; the first draft was no longer this major stepping 
stone, since the children modified their documents continuously. 

Now, one study that showed no benefit of the technology made that claim on the basis of the "first drafts" 
of the children not showing much improvement. Using convergent analysis then, we felt we could now 
explain that negative result. That is, the evaluation used in that study measured the wrong thing; first 
drafts are not the key marker when children use word processors. 

Only by comparing across the literature were we able to ferret out that important observation. And only 
by having the literature in a standardized format were we able to look across the literature. Thus, we feel 
the database of profiles provides a valuable resource for educators and researchers who wish to extract 
value from the research base. To assist in that process, we are now developing computer-based 
visualization tools that will make it easier yet to compare/contrast across studies. 



Practitioner-Oriented Lit Review 



To further help focus our review of the literature, we developed a "practitioners'-oriented literature 
review." That is, in typical literature reviews, authors are still speaking to other reseachers; hence the 
issues they tend to focus on are not necessarily the issues that would assist teachers or administrators. For 
example, the quote below taken from one literature review of the writing education and technology field 
is most illuminating of the problem: the author does not provide suggestions for how teachers who don't 
have advanced students, who don't have pervasive technology, etc. to get around these problematic -- 
and typical -- situations! 

"Fairly consistently, results favored older, more able students, especially those exposed to 
relatively lengthy treatments that were well-grounded in appropriate theoretical 
frameworks." 
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Reed, 1996 

In our literature review, then, we first posed a set of 13 teacher-oriented questions. See Table 1, below. 
These questions were suggested to us by practicing teachers. We then searched the online profiles for 
empirical studies that were relevant to the questions. The resulting "literature review” is available online. 

Extracting Nuggets of Wisdom From the Research Literature 

In what follows we present two examples of nuggets that we have extracted from the research literature 
on writing education and technology using our convergent analysis method. A caveat: while one might 
well want a list of specific pieces of wisdom to serve as prescriptions. If only it were that simple and 
straightforward! The tension is this: to put forward such a list one must abstract away all the nitty-gritty 
details of the situation. That process produces very general statements of which Reed's is a shining 
example: "lengthy treatments" are effective. 

Our nuggets of wisdom, then, tend to be more of a process, a way to examine a teacher question and 
address it based on the literature. This issue will become clearer after we present the two sample nuggets. 

Example 1: It's All About Tradeoffs 

Consider, then, this teacher-oriented practical question: 

"How much time should I spend preparing my students to use a word processor?" 

By looking across all the studies at how each study addressed this issue, the "answer" that can be 
constructed to this question is: "it depends." While on the surface that answer is nor particularly 
satisfying, providing a description of "what it depends on" may well be useful to teachers: 

It does not seem to be the case that there is some hard and fixed minimum amount of time 
on the computer that is needed in order to insure a successful writing experience. For 
example, we did not see that students must have X hours of keyboarding before they start 
writing. Rather, success in writing on the computer could be had from a broad range of 
preparedness activities, from computer literacy training (ie., Eastman, 1989, 324; Beichner, 

1994, 82) to keyboarding (ie., Kurth, 1997, 190; Dalton & Watson, 1986, 207), from a few 
hours (ie., Borgh & Dickson, 1992, 141) on the computer to long-term exposure (ie., Diaute, 

1986, 325/163; Parr, 1994-95, 135; Snyder, 1994, 148; Beichner, 1994, 82; Fais & 

Wanderman, 1987, 275). The research does show that in those projects where students 
tended to have less preparation they had a greater likelihood of obtaining a negative 
outcome (ie., Lohr, et al., 1996, 307). In contrast, in those projects where students had even 
a moderate proficiency coming into the writing activity, there was a good likelihood of 
achieving a positive outcome (ie., McAllister & Louth, 1988, 161; Snyder, 1994, 148; 

Lehrer, et ah, 1994, 80). 

In effect, the amount of time preparing students does not seem to be the determining factor 
for computer-writing success! Moreover, formal keyboard training does not seem to be a 
necessary ingredient. For example, if the treatment is lengthy then students with less 
background will catch up through the extended term of the writing experience (ie., Beichner, 

1994, 82; Parr, 1994-95, 135). If word processing is available to children after school or at 
home, then again, they will catch up if the activity is an extended one. If word processing is 
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used in other subject areas in the curricula, then again, this activity will enable those less 
prepared to catch up with those more prepared. 



In other words, by looking more globally at the research studies, we were able to see how different 
studies used different strategies to accomplish the same goal. Thus there was no one strategy for 
achieving the condition that students go into a writing assignment using a word processor with 
experience using a word processor. The research literature depicts a plethora of strategies. Thus, in any 
given situation, a teacher will have to decide how to manage the tradeoffs, e.g., extend the writing 
assignment if your children are less prepared, take advantage of homework periods after school for the 
less prepared children to become comfortable with word processors, etc., etc., etc. 



The fact that there is no simple, straightforward answer to questions such as the above one is not a bug, 
but a feature! That is, in effect, the literature sanctions teachers to be inventive and to take into 
consideration the local needs, resources, and even idiosyncrasies of their classroom. From our reading, 
the research acknowledges the importance and relevance of the local context; prescriptions that ignore 
that local context implicitly devalue the contributions of classroom teachers towards creating effective 
learning environments. 



Example 2: Technology Changes the Task Which In Turn Changes .... 
Everything! 

Consider, then, another teacher-generated question: 

How do I evaluate the quality of the children's writing when they use a word processor? 

The literature is most interesting on this point. In writing with pencil-and-paper, children create a first 
draft, receive feedback, and then revise it. (Time permitting, there may be additional rounds of feedback 
and revision.) In writing education, one important evaluation measure of a child's work has been the 
amount of change from the first draft to the final draft. 

Now, in studies where children used a word processor, it turned out that when that metric was used, it 
showed that there was not much change between the first draft and the final draft. On the basis of that 
finding, the researchers concluded that word processors were not helping children write more effectively, 
(ie., Owston, et.al, 1992, 164; Owston & Wideman, 1997, 238; Snyder, 1994, 148; Diaute, 1986, 
325/163) 

Again, a more global perspective on the research literature provides a clearer picture of the situation. 
Using convergent analysis and looking over all the research studies what we saw was that studies 
reported that children were constantly modifying their work; in effect, there wasn't a first draft! A word 
processor does make changing a document relatively simple (e.g., in comparison to changing a penned 
document or in comparison to changing a typewriter-produced document). The studies observed that the 
writing process children employed using a word processor was different from the writing process 
children employed with pencil-and-paper technology. While teacher feedback did cause the children to 
revise their word-processed documents, they were revising as they wrote, in response to their own 
thoughts and as a result of conversations with other children who read the documents over their 
shoulders, (ie, Owston & Wideman, 1997, 238) 

This example illustrates how technology changes the nature of what goes on in the classroom: word 
processing technology engenders a different writing process when compared to the writing process using 
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pencil-and-paper technology. Clearly, that change in process had implications for evaluating the 
children's written documents. 



What other ripple effects does this change in the nature of the activity have on the classroom? For 
example, "time on task" becomes more problematic, e.g., are there more comfortable intermediate 
stopping points when using a word processor in comparison to using pencil-and-paper? When should the 
teacher give feedback to children on their word-processed documents? Again, there are no single answers 
that are right for all classrooms. 



Issues Needing Research & Development 

Unfortunately, even a careful reading of the literature will not inform all aspects of classroom practice. 
Research has not typically been driven by the needs of classroom teachers, and thus there are major 
lapses and gaps in the research literature. 

• For example, there is precious little research on how to support children in transferring their 
writing skills to other contexts (e.g., from a writing class that uses computers to a social studies 
test where writing is not done on computers, or even to a social studies class where writing is done 
on computers!). While there were a few studies that demonstrated that transfer from one writing 
task to another was achievable, the number of those studies was small and more importantly, there 
was little discussion on the conditions for achieving that transfer. 

In situations such as the above one, there aren't enough studies to which we can apply convergent 
analysis, and thus we weren't able to tease out the relevant conditions and potential compromises that are 
needed to inform practice. 

What specific topics need to be explored by researchers? For starters, in looking over our 
practitioners-oriented literature review, we see a number of questions that have only a handful of 
references back to the literature. If indeed those questions are important to teachers — and we think they 
are — then these questions suggest areas for further exploration (e.g., transfer, collaborative writing, 
using multimedia for self-expression — are all topics that have little research behind them). 

In addition to more research, new tools to access and analyze the literature are needed. 

• Currently, filling out a Profile is a labor-intensive exercise. But, individuals who have actually 
done profiling, e.g., classroom teachers, report that they came away from reading the literature 
with a much deeper appreciation and understanding of the research when they used the Profile to 
organize their reading. Tools to scaffold that profiling activity therefore would be most useful. 

• Still further, tools that support teachers quickly doing convergent analysis would also be useful. 
Any list of questions will be incomplete; specific teachers and administrators will have particular 
issues that they need input on, and thus they need to be able to quickly and effectively do a 
convergent analysis of the literature. Currently, doing convergent analysis is definitely "an art;" 
what tools will support end-users in making this analysis technique routine? 



Concluding Remarks 

Technology is fast becoming more universal, more pervasive in classrooms. While there are negative 
arguments and naysayers, the trajectoiy is clear: as technology continues to pervade our everyday lives, it 
will do the same for schools and classrooms. The need, the demand for effective ways to use this 

128 

a 

I C I f "Vwww.ed.gov/Technology/TechConf/1999/whltepapers/paper2.html (7 of 10) [4/24/2001 2:38:08 PM] 



Convergent Analysis 



000000 1)01578 



technology in the classroom will only increase. Research can play an important role in providing 
educational practitioners with concrete suggestions on why and how to use technology with their 
students. However, there are real barriers for teachers and administrators in gaining access to the wisdom 
in that research. Currently, research is written with other researchers as an audience; currently, there are 
precious few tools for practitioners to use in accessing research; and there are significant gaps in the 
research since practice has not been a major driver of the research. Towards addressing these challenges 
and extracting value from the research literature, then, we put forward the Convergent Analysis method. 
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Curricular issues 

• Why should I introduce and use word processing in my writing activities in elementary school? 

• Compared to writing with a pencil and paper, in what ways does the use of a word processor 
change the children's writing activities? (e.g., do they still make first drafts then final drafts?) 

• How should I structure the topics the children write about on the computer? 

• What Internet writing activities should I involve my students in and why? How should I use email 
to encourage writing? 

• When might I have my students create multimedia documents and hypermedia documents? 

• There are new computer-based tools coming out that support group writing activities; when and 
how should those be used? 

Instructional issues 

• How much time should I spend on computer skills before I teach my students to write on the 
computer? 

• How can the technology help with special populations (e.g., pre-emergent writers, dyslexia, 
disgraphia and learning disabilities)? 

• How can I to use collaborative groups in my 4th grade classroom to improve writing? 

• How much time should I provide for students to use word processors? (e.g., How many 
assignments per semester is good? How much time should children to spend at one sitting?) 

• How can I prepare children to transfer their writing skills from the computer to paper & pencil? 

• How do I prepare my students to move from a Mac word processor, in 2nd grade, to a Windows 
word processor? 

Infrastructural issues 
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• How can I use a word processor for writing with 1-3 computers in my classroom? How can I use 
the computer lab, with 1 5 computers, to compliment my classroom computers? 
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Abstract: This paper discusses methods for observing changes in classroom processes in 
project-based classrooms using multimedia technology. The tool was used as part of a 
five-year evaluation of a local Technology Innovation Challenge Grant program called 
Challenge 2000: Multimedia Project. In the paper, we discuss the design of the observation 
tool and present findings about the differences in classroom processes between Multimedia 
Project classrooms and comparison classrooms. Project classrooms, we found, are more 
likely to be learner-centered and engage students in long-term, complex assignments. 

INTRODUCTION 

Case studies such as from Apple's Classrooms of Tomorrow (Sandholtz, Ringstaff, & Dwyer, 1996) and 
from reforming schools across the country (Means & Olson, 1995) point to the potential of new 
technologies to support new ways of teaching. Moreover, these studies have provided rich details about 
what takes place in classrooms as local technology-supported reforms are implemented. 

In addition to these kinds of studies of technology use in schools, in recent years there have been a 
number of published reports using national data that show the promise of technology to support school 
reform. Many of these studies rely on survey data, student achievement data, or a combination of both in 
their analysis. Becker (1999), for example, has published some of the results from his recent survey of 
teaching practice and technology use in the United States on the World Wide Web, in which he found 
fewer than 1/3 of teachers were having students conduct research on the Web. The Educational Testing 
Service has found that when students use computers to apply higher order concepts and when teachers 
are knowledgeable about how to use computers as productivity tools, students show significant gains in 
mathematics achievement (Wenglinsky, 1998). 

As initiatives and programs that use technology to help drive school reform proliferate, there is an 
increasing need for tools that can help measure whether these programs achieve anticipated changes in 
teaching practice. Certainly tools used in both case study research and large-scale studies can be used or 
adapted for use in evaluation, but researchers must pay careful attention to local program contexts when 
considering how to use these tools. First, evaluators must first examine the program or initiative's 
specific design; that is, how interventions are expected to bring about particular changes in teaching 
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practice. The anticipated changes must themselves be described in enough detail for researchers, other 
observers, and teachers themselves to be able to know when they’ve achieved those desired changes in 
teaching practice. Second, evaluators must consider a range. of factors related to the opportunities and 
constraints of the evaluation process itself: What is the scope of the evaluation? Who are the stakeholders 
and partners in the evaluation? What kinds of data are needed to evaluate the design? How will the data 
be analyzed? How will different partners in the initiative use the data? 



In this paper, we describe how researchers at SRI International’s Center for Technology and Learning 
designed and used an observation tool as part of its evaluation of a local Technology Innovation 
Challenge Grant. The paper is intended to present both a process and a tool that researchers, program 
designers, and teacher-researchers might adapt to similar programs and initiatives. First, we describe both 
the program's theory of action and some of the important opportunities and constraints in the evaluation 
design. Next, we describe the design of the observation tool itself and how it was used over the course of 
two years. Finally, we present some of the key findings from the observation study, with specific 
attention to changes in teaching practice that were observed. 



THE CHALLENGE 2000: MULTIMEDIA PROJECT DESIGN 

The context of this study is a project funded through the U.S. Department of Education's Technology 
Innovation Challenge Grants program. The grant Was awarded over four years ago to the San Mateo 
County Office of Education and is jointly coordinated by the county office and Joint Venture: Silicon 
Valley, a partnership of area businesses focused on improving the quality of life in the Silicon Valley 
region in northern California. The federal grant funded the Challenge 2000: Multimedia Project and was 
designed to support Joint Venture's larger education reform initiative, which is aimed at making Silicon 
Valley students among the most sought-after by employers in the region for jobs in the new workplaces 
of the twenty-first century. 

The Challenge 2000: Multimedia Project aims to engage students in their own learning and develop 
students' skills of collaboration, decision-making, and complex problem solving. To accomplish these 
goals, the Project has adopted a model of Project-Based Learning using Multimedia (PBL+MM) and 
provided supports to teachers in learning how to implement projects and use technology effectively to 
enhance and support student learning. By implementing student-centered projects and providing supports 
to teachers, it is expected that classroom processes and teaching practice will change, leading to better 
outcomes for students. 



Student-Centered Projects 

The model of project-based learning using multimedia is a research-based model developed by the 
Challenge 2000 participants in collaboration with researchers from the Institute for Research on 
Learning. This model incorporates all of the dimensions that have been traditionally associated with a 
project approach to learning (Blumenfeld, et al., 1991; Kirkpatrick, 1918; Rawcliffe, 1925), such as 
having a real-world connection, but adds the practice of producing final projects in a multimedia fomiat 
as a central part of the practice. Among the kinds of multimedia products that students have produced are 
HyperStudio stacks, Web pages or sites, PowerPoint presentations, animations and videos, and music 
CDs. 



There are seven components of the Project Based Learning Using Multimedia model. Projects are 
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expected to: 

; • Be anchored in core curriculum; multidisciplinary 

• Involve students in sustained effort over time 

• Involve student decision-making 

• Be collaborative 

• Have a clear real-world connection 

• Use systematic assessment: both along the way and end product 

• Take advantage of multimedia as a communication tool 



It is important to note here that the seventh component, use of multimedia technologies, is not conceived 
as a stand-alone component. Multimedia technologies are intended to be used as tools in the planning, 
developing, and presenting projects. It is believed that the power cf multimedia lies primarily in the 
extent to which it is integrated within the goals of the project and ongoing curriculum for the class. 
Products that students create come to serve as public artifacts (Allen & Pea, 1992; Blumenfeld, et al., 
1991; Penuel, Cole, Korbak, & Jump, under review) that are part of the classroom community's memory 
of what it has accomplished. 



Teacher Supports for Technology Use 

To help teachers implement the PBL+MM model in their classrooms, Multimedia Project staff have 
created a number of supports and incentives for teachers. The project's theory of action has emphasized 
the importance of creating a learning community among participating teachers. Initially, the Institute for 
Research on Learning provided training on how to plan and implement projects in the classroom and on 
how to use multimedia technology. As the project developed, participating teachers formed a cadre (a 
Project name) that took on more and more responsibility for planning and conducting their own 
professional development. As teachers have become more practiced and skilled in implementing the 
model, they have refined it and shared it with new participants that join the Project. 

Multimedia Project teachers establish a peer community of learners in which they gradually take on 
responsibility for planning and conducting their own professional development. Veteran teachers share 
their skills with less experienced colleagues. Many of these veteran teachers serve in special roles funded 
in part through the Multimedia Project. Technology Learning Coordinators in the project are skilled in 
both the use of technology and innovative teaching practice and are available to teachers in the project 
for help. Typically, a portion of the Technology Learning Coordinators' time is also spent providing 
technical assistance to teachers experiencing problems with specific technologies. 

The Project also provides a system of recognition and rewards for project teachers. Teachers may apply 
individually or as partners for mini-grants, allowing them to purchase equipment, software, peripherals, 
and/or training if they implement multimedia projects in their classes. Providing greater access to 
hardware, software, and to the Internet has made it possible for students to complete projects they would 
never have been able to do before the grant. Teachers may also be recognized for their students' 
contributions to annual Multimedia Fairs held by the school teams participating within the Challenge 
2000: Multimedia Project. 
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In the Multimedia Project design, the implementation of student-centered projects and development of a 
pefcr learning community with access to technology and technical support are expected to result in 
changes in what goes on inside project classrooms. Specifically, the design calls for classrooms in which: 

• students engage in longer-term, more complex assignments 

• teachers act as coaches and facilitators of student learning 

• students engage in more small-group collaborative activities 

• there is greater involvement with external resources, including heightened attention to external 
audiences for student work 

In turn, these changes in classroom practices are expected to bring about the student outcomes described 
above-greater skills in collaboration, decision-making, and complex problem solving. 

The changes in classroom processes can be seen as intermediate outcomes of the Multimedia Project, 
benchmarks that can be used to indicate that the project is progressing toward meeting its objectives for 
student learning. Because of the central importance of changed classroom processes and teaching 
practice in the program design, we chose to develop an instrument to help measure whether in fact 
participating classrooms were in fact more student-centered, collaborative, and engaged with external 
resources. In the next section, we describe the protocol and the evaluation context in which this 
instrument was designed. 

DESIGNING AN OBSERVATION PROTOCOL TO MEASURE 

CLASSROOM PROCESSES 



The Evaluation Context 

SRI International (SRI) is under contract to conduct the evaluation of the federal grant coordinated by 
Joint Venture: Silicon Valley and the San Mateo County Office of Education. The evaluation is a 
multi-method, five-year study of the implementatioi and outcomes of the Multimedia Project. As the 
project has developed over its first four years, the evaluation has moved from a primary focus on 
documenting implementation (describing student-centered projects and teacher supports) to measuring 
outcomes of the project. Research questions evolved as the project model has crystallized: each year, 
evaluators are able to ask more focused questions about teaching and learning in Multimedia Project 
classrooms. 

Throughout the project, SRI International has adopted a partnership approach to the evaluation process. 
Staff from SRI serve on the Multimedia Project's Coordinating Committee, which meets monthly to 
discuss the progress of the Project's activities. The Committee consists of representatives from San 
Mateo County Office of Education, JVSV, the Institute for Research on Learning, and other key program 
partners. At these meetings, SRI presents information about how teachers and students are participating 
in and responding to various project activities. Other members of the Coordinating Committee, led by 
JVSV, identify their own questions that in turn shape each year's evaluation design. 

SRI has used case studies, interviews, teacher surveys, classroom observations, school-wide indicators of 
achievement, and performance assessment data as part of the study. Each of these methods has been used 
either to document implementation of the project or measure progress toward outcomes. 
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Year 3 Study Findings 

Initially, the observation protocol was adapted from one used by researchers at the National Center for 
Research on Evaluation, Standards, and Student Testing (CRESST) for use in the studies of Apple 
Classrooms of Tomorrow in the third year of the program in part to provide evidence to program 
stakeholders that the Multimedia Project was making progress toward changing classroom processes in 
project classrooms. 



In this first study, 19 classrooms were chosen from among Challenge 2000 classrooms across grade 
levels for observation in the fall of 1997 and the spring of 1998. Principals from schools where SRI was 
conducting case studies nominated three technology-using and three non-technology-using classrooms 
for participation. In most cases, these schools had three teachers participating in the project, or otherwise 
engaged in technology use, but some did not. In those cases, additional non-technology-using classrooms 
were observed for the study. The original observation protocol examined variables such as the dominant 
classroom activities, teacher and student roles, the nature of ongoing student work, and the level of 
student engagement. 

The results of the study showed significant changes in classroom processes from fall to spring, with 
differences between technology-using and non-technology-using classrooms (Means & Golan, 1998). 

For example, in the fall students in technology-using classrooms were only slightly more likely than 
students in comparison classes to be engaged in long-term projects at the time of the observation. By 
spring, that gap was very wide, with 67% of technology-using classrooms versus 14% of non-technology 
using classrooms involved in extended projects at the time of observation. Similarly, teachers from both 
sets of classrooms were equally likely to be engaged primarily in questioning students, a traditional role 
for teachers, in the fall. In the spring, far fewer technology-using teachers used questioning as their 
dominant way of relating to students (7% versus 49% for non-technology-using teachers). Instead, 
technology-using teachers were much more likely to be in a helping or monitoring role within the 
classroom (43% in the spring versus 1 8% of non-technology-using classrooms). 

Similarly, students in technology-using classrooms were much more likely than their peers in 
non-technology-using classrooms to be engaged in constructing products and working in small groups in 
collaborative activity. Again, the differences were much greater in the spring than in the fall. In the fall, 
56% of technology-using classrooms involved students in constructing products compared to 39% of 
non-technology-using classrooms. By the spring, that gap widened: 73% of technology-using classrooms 
engaged students in constructing products versus 38% of non-technology-using classrooms. While in the 
fall, few classrooms from either sample engaged students in small-group collaboration, nearly a quarter 
of technology-using classrooms involved small-group collaborative activity in the spring (compared to 
0% of non-technology-using classrooms). 



Adaptation of the Protocol for Year 4 

A multi-year evaluation affords the opportunity to revise instruments and processes based on what is 
learned from using them and based on the purposes the instruments serve in the overall evaluation design 
for that particular year. At the beginning of Year 4, we made some revisions to the observation protocol 
itself and to the data collection and analysis process. 



In Year 4, the evaluation data collection activities focused increasingly on measuring outcomes from the 
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Multimedia Project. A performance assessment task was created to measure student skills in design, 
collaboration, and mastery of content. We decided to link the planned replication of the observation 
study with the performance assessment task. In this way, we could test the design or conceptual 
framework of the model by answering the question: Do changes in classroom processes lead to different 
levels of student performance? At the time of writing, performance assessment data are still under 
analysis. In this paper, we describe the design changes to the observation protocol and report results from 
Year 4, which continue to point to the promise of the design in changing classroom processes. 



Selection of Classrooms 

In Year 4, the classrooms were selected using a different method in order to ensure that a large number of 
veteran teachers would be included in the sample of Multimedia Project classrooms. For this reason, we 
do not use the tenns technology-using and non-technology-using classrooms to characterize the two 
samples (though the two samples of classrooms can be distinguished in this way). Rather, the study is 
comprised of 12 project classrooms and 9 comparison classrooms. As in Year 4, observations were 
conducted once in the fall and in the spring, both times within a three- week window. 

Project classrooms selected for the study were a combination of experienced and novice teachers within 
the Multimedia Project who were funded with mini-grants for the 1998-99 school year. Principals from 
the project teachers' schools selected comparison classroom teachers. Principals were given instructions 
to select a teacher in the same grade who was not a part of the project but who taught in a subject area 
similar to the project teacher. Because the project encouraged partnerships within schools, finding a 
comparison teacher at the same grade level was not always possible. In two cases, classrooms from the 
same grade level at a comparable school in the same district were chosen. Still, the resulting classrooms 
were similar in size and in demographic composition (Table 1). 



Table 1. 



Composition of Classrooms in the Study 





MM Project Classrooms 


Comparison Classrooms 


Average attendance 


27.5 % 


28.4 % 


Ethnic composition 






White 


56% 


61 % 


Asian/Pacific Islander 


20% 


17% 


Latino 


15% 


16% 


African American 


2 % . 


4% 


Other 


7% 


2% 



Classrooms did differ on one significant measure, namely the number of computers that were in their 
classrooms. On average, Multimedia Project classrooms had 6 computers, while comparison classrooms 
had only 2. 

In addition, both samples included only 6 th and 7 th grade classrooms. We selected classrooms from these 
two grades because the performance task designed to measure the impact of the project on student 
learning was targeted to middle grades students. 
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In Year 4, we added two sets of items to the protocol that have been emphasized by sociocultural 
researchers (Cazden, 1988; Lemke, 1985; Mehan, 1979; Wells, 19xx; Wertsch, 1991) as important for 
sustaining extended student inquiry. We asked observers to characterize the different forms of discourse 
that students and teachers used in the classroom. For example, observers looked for "instructional 
questions" (Mehan, 1979; see also Heath, 1983) in which teachers ask brief questions of students, to 
which the answer is already known, to test students knowledge of isolated facts. In general, we were 
interested to know whether Multimedia Project classrooms engaged in what have been called more 
dialogic (Bakhtin, 1981) forms of discourse than comparison classrooms. By dialogic, we mean forms of 
discourse that engage students and teachers in discussions that are not always teacher-controlled. By 
contrast, we anticipated that comparison classrooms might be more likely dominated by a monologic or 
lecture-oriented form of discourse. 

We also wanted to be able to analyze better the extent to which teachers allowed students to work 
independently with limited strategic assistance (Wertsch, 1985). We expected teachers in project 
classrooms be more inclined than those in comparison classrooms to allocate more time than comparison 
classrooms to having students practice learning skills on their own, rather than simply demonstrating the 
skills to students or telling them about what they need to know. We predicted that teachers would provide 
assistance as needed in project classrooms, but students would be given primary responsibility for their 
own learning. 

Activity as Unit of Analysis 

Consistent with a sociocultural approach to observing classroom practice, we also chose to use activity 
rather than observation time as the primary unit of analysis for our Year 4 observations. For purposes of 
the study, an activity is defined as student engagement in some kind of educationally relevant product. 
Those products include: a story written, a reading completed, a topic discussed, science observations 
made and recorded, a set of related problems at the board worked through, a pre-writing activity 
completed, a painting painted, et cetera. Sometimes an activity produces no tangible product (students 
listen to a lecture) but the activity is nonetheless organized to produce a definable outcome-e.g. coverage 
of a particular topic. 

Operationally, we defined the activities as different when two or more of the following changes took 
place in the classroom: 

• A new product or objective is introduced by the teacher or other students that is followed by new 
patterns of thinking, communicating, and acting. 

• The topic changes, whether signaled by movement from one subject area to another or to a 
different domain within a particular subject. 

• The activity or participant structure changes; in other words, the way roles are assigned among 
students or the ways students and teachers are interacting shifts (e.g., from whole group lecture to 
small group collaboration). 

• The spatial arrangements in the room shift, in that either people change places or physical objects 
in the room are re-configured to afford a different kind of activity. 

• The teacher (or students) makes a bid to close a segment of classroom activity, signaled by specific 
instructions to students about "wrapping up" or by teachers beginning to review just finished work 
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or instructing students about re-arranging space in the classroom. 
Multiple Levels of Analysis 



By selecting activity as the unit of analysis and recording the amount of time spent on each activity, 
additional avenues for data analysis were opened up. Whereas in Year 3, observers recorded what was 
happening in three fifteen-minute intervals throughout their observations, in Year 4, observers recorded 
anywhere from between 1 and 4 different activities across a forty-five minute observation period for each 
classroom. The Year 4 data permit analysis of the amount of time spent in various activities, by the 
' dominant activities within classrooms, and by classroom teacher. In this paper, we present analyses from 
the first two dimensions, time and activity. 



OBSERVED DIFFERENCES BETWEEN PROJECT AND COMPARISON 

CLASSROOMS 



In this section, we report a number of significant differences we found between Multimedia Project 
classrooms and comparison classrooms. We consider the extent to which project classrooms were more 
focused about long-term projects, learner-centered, collaborative, and oriented toward people and 
communities outside the classroom. 



Engagement in Long-Term, Complex Assignments 

Time Spent on Long Term Assignments. Students in Multimedia Project classrooms engaged in 
significantly longer activities (p<.05) than students in comparison classrooms. Moreover, they were more 
likely to be engaged in long-term activities-that is, activities that spanned more than a week of class 
time-than their counterparts in the comparison classrooms. Moreover, both in the fall and the spring, 
students spent more time in project classrooms engaged in long-term activities that lasted a week or more 
(an average of 84% of the time in project classrooms versus 49% of the time in comparison classrooms). 

Analysis of the Complexity of Activities. For the Multimedia Project to be successful, it would not be 
enough to say that students are engaged in long-term activities. Something of the quality of their activity 
would need to be observed and understood to argue successfully that projects were transforming 
classroom processes. An analysis of student actions reveals that in fact, students are engaged in complex, 
cognitively challenging tasks in project classrooms. 

Students were observed to be engaged in more of what might be called the cognitive activities of design. 
In other words, they were engaged in the kinds of higher-level cognitive activities characteristic of 
multimedia design as described by Lehrer (1993): deciding on the structure of a presentation; creating 
multiple representations, models, and analogies; arguing about or evaluating information; thinking about 
one's audience; and revising or editing work. Table 2 shows the differences between Multimedia Project 
and comparison classrooms for those activities marked "dominant" by observers. 



Table 2. 

Dominant Activities Observed 



MMP Classrooms , 


Comparison 




Classrooms 
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Teacher-directed solo 
activities (e.g., reading 
silently, listening to teacher) 


13 


23 


Cognitive activities of design 
(e.g., deciding on structure of 
a presentation) 


13 


3 



c2=9.03, df=l,p<.01 



Teachers as Coaches and Facilitators 

Time Spent in Independent Activity. In Multimedia Project classrooms, more time was spent having 
students practice skills on their own (whether independently or as a group) with strategic assistance 
provided by teachers as needed, than having students watch or listen as teachers performed a task for 
them or explained a process to them (See Figure 2). This difference was particularly pronounced in the 
spring, when teacher-led activities comprised 29% of time in project classrooms versus comparison 
classrooms (62%). It is clear from these data that project teachers are more likely to give major 
responsibility to students for their own learning than do comparison teachers. 

Dominant Roles of Teachers. Teachers in Multimedia Project classrooms were much more likely to be 
engaged in facilitative roles within classroom activities than were teachers from comparison classrooms. 
In other words, they were more likely to be engaged in assisting or helping students by moving about the 
classroom and responding to student questions or providing help when they see a need for it. This 
facilitative role is evident in the greater extent to which teachers help to organize the process by which 
students can work productively on their own, whether in groups or individually. By contrast, the 
dominant role of teachers within comparison classrooms was more directive. Teachers were more likely 
to be explaining concepts, providing information, or questioning students about their understanding of 
material (see Table 3). 



Table 3. 

Dominant Teacher Roles Observed 





MMP Classrooms 


Comparison Classrooms 


Directive Role 

(e.g., explaining concepts, providing 
information, questioning students) 


11 


17 


Facilitative Role 

(e.g., assisting or helping, managing 
the organization of the task, 
monitoring as students work on their ; 
own) 


13 


7 
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c2=7.81, df=l, p<.05 



Engagement in Small-Group Collaborative Activity 

Time Devoted to Small Group Activity Students in Multimedia Project classrooms were more likely than 
comparison students to spend time engaged in small group collaboration. T his collaboration was 
supported, moreover, by discourse patterns that allowed students to direct discussion among their peers 
about the content of the class. 

While in the fall, students spent roughly the same amount of time in project and comparison classrooms 
engaged in small-group discussion, by the spring, project classrooms devoted much more time to this 
form of discourse. A corollary finding is that by the spring time, only 3% of the time in project 
classrooms was devoted to "instructional" or known-answer questions compared to 72% of the time in 
comparison classrooms. 

Analysis of Dominant Activities. An analysis by activity yields similar results. There was a more dialogic 
pattern of discourse within project classrooms than within comparison classrooms in the spring. By 
dialogic, we mean forms of discourse that engage students and teachers in discussions that are not always 
teacher-controlled (e.g., lecture). By contrast, comparison classrooms were much more likely to be 
observed as having a monologic or lecture-oriented discourse dominate classroom time (c2=7.88, df=l, 

p<.01). 

Involvement with External Resources 

Time Spent Using the Internet. One of the most valuable tools for connecting classrooms to wider 
communities is the Internet. By the spring, students in project classrooms spent half of the time observed 
using the Internet, searching for information, graphics, pictures, sounds, and other material to use for 
their multimedia presentations. The Internet was not used at all in comparison classrooms, either in the 
fall or the spring. 

Attention to an External Audience. Yet another way that classrooms are connected to broader 
communities is through the student-led projects themselves, which typically have an audience outside the 
classroom. In this respect, project classrooms differ significantly from comparison classrooms in the 
likelihood that students will be engaged in discussion about how their audiences would respond to 
aspects of a product being produced (Fisher Exact Test, p<.001). In spring, 35% of the activities in 
project classroom involved students considering the audience of their work, whereas none of the 
activities observed in comparison classrooms found students attending to the audience of their work 
(beyond the teacher-as-audience). 

Discussion and Implications 

In each of the dimensions we observed and analyzed, Multimedia Project classrooms distinguished 
themselves from comparison classrooms by being significantly more student-centered and organized 
about the collaborative construction of complex products. These findings not only constitute evidence of 
the projects success in stimulating desired changes at the classroom level at classroom, but also measure 
power of our instrument to measure and capture these changes. 

At the same time, this effect was not evident throughout the whole school year on all dimensions. As in 
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Year 3, classrooms were much more likely to be engaged in focused efforts to complete student projects 
in-the spring than in the fall, even though care was taken to select only Multimedia Project teachers for 
the sample. The only dimension in which project teachers differed from comparison classrooms 
throughout the school year was in the amount of time students spent engaged in small-group 
collaborative activity. 

There are a number of possible reasons why we observed this time-of-year effect. First, one could argue 
that project teachers are among the most innovative teachers within their schools and were predisposed to 
become a part of the Multimedia Project. The project, therefore, might not be the cause of the difference 
in classroom processes. This interpretation is not consistent with case study and interview data, however, 
in which many project teachers describe how the project has changed their view and practice of teaching. 

Another possible interpretation of these data are that even among project teachers, there is a natural 
building of component skills for projects that takes place in the fall. In the fall, many project teachers use 
time to teach students research skills or how to use different multimedia software packages. The teachers 
are still focusing on projects, but their work requires much more direct, teacher-led instruction at this 
stage. 

One third possible reason why we have observed this time-of-year effect is the timing of the Multimedia 
Fairs. The fairs are held in the spring each year and motivate much of the activity of the project teachers 
and their students throughout the spring months. At the time of our observations in April, many of the 
project classrooms were in the middle of working on projects they would show at the fairs. These fairs, in 
turn, motivate students' attention to an external audience, since people from other schools and the 
community will see their work. 

Even with this school year effect, a convincing case can be made that project teachers are more likely to 
engage students in small-group collaborative activity, regardless of whether they are working on their 
multimedia projects. In some cases, it may be that small group work is part of the school's philosophy, 
and the emphasis on collaboration cannot be attributed solely to the work of the Multimedia Project (see 
Penuel, Cole, Korbak, & Jump, under review). Still, the success of previous student projects appears to 
contribute to teachers' eagerness to use collaborative learning as a tool to promote greater mastery of 
content and skill in working well with others. 

Overall, the results suggest that the project is meeting its objective of transforming classroom processes 
so that they become more student-centered, especially while students are engaged in project-based 
learning using multimedia. The results suggest a strong role for the projects themselves and for the 
Multimedia Fair in contributing to these changes, since differences between project and comparison 
classrooms are much more evident in the spring than in the fall. Indeed, it may be that events like the 
fairs, which provide concrete links between the classroom and other classrooms and the community, 
deserve a more important place as levers for changing classroom practice. 
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The vision is enticing. Computer technologies become the norm in schools that are equipped with 
multimedia, graphics and animation, access to Internet and hand-held and remote devices. There is 
seamlessness of learning activities among home, school and community settings. Students use 
technologies like they use pencils, books and manipulatives to learn content in all of the subject areas. 
Learning goes beyond skills and facts, and students develop thinking and problem solving skills. The 
world is their classroom. In this vision, technologies help students gain mastery of content areas and zip 
at speeds of the fastest Internet connection well beyond and above the standards. Computer technologies 
are the norm rather than the exception, and they become enablers rather than another subject to be taught 
in school. 

Where are we in relation to this vision? After two decades of computers in classrooms we can say there 
have been some major strides. Most schools have computer labs; many schools have computers in every 
classroom. Over 90% of schools are wired (connected to the Internet), and over one-third of teachers 
have Internet access in their classrooms, which they and their students use frequently. Most teachers and 
students use word processing programs. We see teachers who use spreadsheets, simulations, CAD 
systems and multimedia software, but then again, we are especially tuned into looking and finding 
exemplars of technology use in schools. We know that a variety of factors predict whether and how 
teachers will use technology, including access, training, teaching philosophy, and collaboration with 
other teachers . 

Still, broad statistics do not tell the whole story. Are computer technologies transforming classroom 
teaching and learning? Are they making it possible for students to achieve standards and go beyond? We 
don't have answers to these questions, yet we can report on some of the trends we see as a result of our 
work in the field. 

IRL has been experimenting for many years with how technologies can leverage learning. In our projects 
we have spent much time introducing teachers to technology, developing technologies for content 
integration, and researching the process and practices of the teachers and students along the way. We 
have worked with a wide range of teachers, some of whom wanted to try computer technologies, and 
some who felt obligated to try. While we have seen many demonstrations of the content learning we 
know is possible, we have not seen large-scale adoption of technology in the core subject areas. 

Even in schools where there is a strong push to adopt and use technologies, the road to content 
fulfillment is a long one. We see a pattern where the technology is front and center stage, rather than the 
academic content. In case after case we see that when computer technologies are adopted, the learning 
about the technology often takes over, and it is only after several rounds of integrating technology with 
content that content emerges in strong ways. The technology learning curve tends to eclipse content 
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learning temporarily - both kids and teachers seem to orient to technology until they become 
comfortable. This dilemma has important implications for teachers' willingness to adopt technology. This 
is because teachers in core subjects rightly see content, not technology, as the primary focus of their 
teaching efforts. Teachers' attention to content is important to pedagogy and usually leads to workable 
solutions. 



The good news is that content learning does emerge and is very rich once the technology recedes as the 
focus of activities in the classroom. At its best, technology can facilitate deep exploration and integration 
of information, high-level thinking, and profound engagement by allowing students to design, explore, 
experiment, access information, and model complex phenomena. 

Our research also indicates that while infusing technology into schools is worthwhile, it can be a long 
road from promise to reality. 

• Content integration takes time: Teachers' first technology projects generate excitement, but often 
little content learning. Often it takes a few years until teachers can use technology effectively in 
core subject areas. Initially, teachers and students don't expect much content in technology projects 
and are satisfied if projects are completed and look good. Teachers learn to use computer 
technologies and leam how to bring content learning to the forefront with, in some cases, 
impressive results on the part of the students. Teachers eventually leam to view the learning 
process in concert with their new technologies and come to understand the ways content 
interactions can be approached. 

• Glitches galore: The bumps in the road to technological competence almost guarantee that 
technology will take center stage over content at first. Inexperienced teachers tend to 
underestimate the time and complexity of a technology-based project. Software glitches and poor 
student work habits (e.g. forgetting to save work) can cause huge delays, often meaning that the 
project has to end just as students are starting to leam some subject matter. 

• Flash over substance: Students and teachers alike are excited by the presentation capabilities of 
the new media, resulting in the "flash over substance" phenomenon. Over and over, we see that 
academic content is allowed to slide initially in a technology-infused project, as students spend 
their time exploring software capacity for special effects and animation. 

Throughout this technology adoption process, teachers tend to worry about content, feel accountable for 
it, and notice when it is missing. This is a key dilemma in the technology adoption process. Teachers 
respond to this dilemma in at least three ways: 

• Back off: Teachers diminish or stop technology use temporarily to make sure students accomplish 
content. This strategy has worked for teachers, but it usually means that computers, relegated to 
the sidelines, are employed for supplemental work, special projects, and skills and practice work 
rather than core subject matter. 

• Keep it simple: With this strategy, teachers stick to one tried and true technology or use only the 
technology capabilities with which they are comfortable. With this strategy you might see a 
teacher encourage writing with word processing, using spreadsheets to make charts and graphs, or 
encouraging students to create reports using presentation software. The teacher might set goals for 
learning new software in the summers and plan to incorporate it in one project until a comfort level 
is reached. While this approach puts a floor on learning (students get some access to technologies), 
it can also impose a ceiling by limiting exploration and scope. 
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• Dive in: A third strategy is to plunge in head first with students using computer technologies, 

. ‘ hoping that teachers and students will learn together about technology. Teachers who use this tend 
to have a lot of trust in their students' abilities to solve problems and find their way to subject 
matter. Sometimes this works, but often students and/or content falls through the cracks. 

With the right support and access, all of these problems tend to recede as teachers and students gain 
experience with technology. Teachers leam how much to structure students' access to content. They 
develop effective assessment tools that help students focus on subject matter. Students come to 
understand the possibilities and expectations for learning with technology. Teachers and students both 
leam that different technologies offer different affordances and constraints in relation to what is being 
learned. They come to know that there are many ways to express subject matter with technology, and that 
technology won't (and doesn't need to) do the whole job. 

To further embellish our characterization of this critical tension between technology learning and content 
learning, we offer cases from two of our projects. In both cases we have seen teachers handle the 
technology/content dilemma and move content leaning to the foreground of activities with computer 
technology. 

The "Where’s The Math?" Problem 

We encountered these issues directly in our Middle-school Mathematics Through Applications Project 
(MMAP). One of MMAP's accomplishments was that it found a working balance between content 
learning and engaging with technology that made it possible for many students to achieve middle school 
math standards. In our description we aim to reveal what occurred between the onset of the content crisis 
recognition to the achievement of true integration. 

The MMAP project created technology-integrated environments where students could participate in 
mathematics learning through designing solutions to real-world problems. The students role play 
architects, population biologists, encryption experts, and analysts of geographic databases who are asked 
to design solutions for various clients. They are equipped with many adult-like computer and 
mathematical tools. We developed and field-tested four software environments and related design-based 
curriculum units. Our first software environment was ArchiTech, a mini-CAD system where students 
could design a floor plan for a structure, manipulate certain variables relating to indoor and outdoor 
temperature and building insulation values, calculate area, perimeter and heating and building costs, and 
analyze the data to make design decisions. We designed the program to be a simple and easy to use. It 
can run on any computer that a school might have. Our hypothesis was that if the software was simple to 
leam and easy to use, teachers and students would be able to enjoy the environment and concentrate fully 
on the mathematics and design tasks presented in the units. 

We wrote a curriculum unit called The Antarctica Project where students design a research station for 
scientists who are going to work in Antarctica. Before releasing the unit into classroom field tests, we ran 
through the unit with our staff, as well as a group of 16 local middle school teachers who worked with us 
over the course of the project, and small groups of middle school students. All learned how to use the 
program quickly and were engaged in the mathematical work as we had hoped. Even teachers who were 
skeptical were impressed with the amount and sophistication of the mathematics that they found 
themselves engaging with as they worked through the design project. They were also impressed with 
how engaging the software was. Our videotapes and observations of these formative trials were 
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confirming as well. The Antarctica unit was written with a notion that teachers would easily (almost 
naturally) identify and track when the time was right to introduce new mathematics concepts, activities, 
skills, or next project steps. Math opportunities would emerge from the designs students created in 
ArchiTech, which would be a focal point around which mathematical engagements, activities, and 
conversations would develop. We hoped that the mathematics would be obvious and ubiquitous. 



The rough and tumble life of classrooms revealed a different reality. While teachers and students were 
engaged, on task, excited and involved with the ArchiTech environment and with the design of the 
research center, it seemed to both teachers and students that they had "fun" with the software and design 
task, but ignored their mathematics work. The students told us they loved their new math class, because it 
was fun to use the computers and great to pick up real life architectural skills. When asked if they were 
learning any math, they looked blankly at us, "No math, but we're learning about the real world." Back 
with the videotapes at IRL, we found students wrestling with scale and proportion problems related to 
their research center designs for thirty minutes at a time. They also analyzed the complex relationships 
among variables in their designs, such as the costs of heating and the insulation values. But the students 
could not identify the math or depict themselves being mathematical, and the teachers were uncertain 
about what math was actually accomplished by the students in the groups. By the time we analyzed the 
tapes and described the math, it was, relative to classroom realities, irrelevant. We asked the question, "Is 
math really being accomplished if no one in the classroom can see it?" Our answer was "No." 

We had a crisis on all levels. The software seemed to be doing its job. It was easy to learn, easy to use, 
engaging to all of the students and provided many opportunities for mathematical content engagement. 

On videotape, we could find children working hard during group time yet reporting they did "nothing" 
mathematically. In project presentations, most students presented lists of rates and costs of specific 
design variables. When we questioned students we found they were capable of talking in quite detailed 
ways about the math they had accomplished and used many representations of their ideas in their 
explanations to us. In general, the students had extremely limited ideas about what constituted 
mathematics, and we decided the problem wasn't only about technology presence. We realized we had 
gotten deep engagement with the technology and gloss engagement with mathematics content. The 
teachers lamented that even though they would sometimes have remarkably complex math conversations 
with children, they were hard pressed in meetings with parents to say what the children had learned. 
Imagine the students telling their parents that math class was great because they had fun playing on the 
computers, and they were learning what it was like to be architects, even though they didn't do any math. 

This pattern was repeated in a second classroom. Both teachers were enthusiastic and committed to doing 
the project the next year. We considered taking a wait-and-see attitude on how the content would play 
out the second time around. However, being a mathematics project, we felt we should alter the approach 
to mathematics content to force a balance. A much higher level of productivity for both teachers and 
students on the mathematics front was needed. We turned our attention to strengthening the unit activities 
by structuring problems with systems of constraint, and embedding specific unit activities and 
assessment tools for enhancing mathematics participation. A variety of activities was added that 
structured students' noticing, naming, further developing and reflecting on the math they encountered in 
their project design work. We also helped teachers make more productive use of their informal 
conversations with students. We found that by encouraging teachers to slow down and spend a few 
minutes with each group, they were able to let students describe their designs, discuss issues or problems, 
interrogate around relevant math topics, and suggest next steps. 
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We saw much more balance after consciously marrying the technology environment to the content. 
Students still engaged with each other, the computer environment and with the mathematics at deep 
levels. Students still felt they were learning how to use important, adult-like tools, learning about adult 
work and problem-solving, yet they also knew they were learning about scale and proportion and using 
and relying on representations of function and variable while making design decisions. 



The message: Activity structure is one way to mediate the interplay between engagement with content 
and immersion in technology environments, so that content is not relegated to the background. Computer 
technologies, like other technologies, are powerful tools for accessing complex mathematical ideas and 
concepts. Technologies can be extremely powerful, provided we take the time to embed them in 
content-rich activities. 



The Challenge 2000 Multimedia Project 

We have addressed similar issues in our work with teachers in the Challenge 2000 Multimedia Project. 
This project has a seven-component model for project-based learning using multimedia that has been 
successful in helping teachers juggle the multiple demands of developing students' subject-matter 
knowledge while teaching technology and collaboration skills. The model suggests that students, engage 
in multi-media-supported projects that have these seven characteristics: 

1 . Anchored in core curriculum; multidisciplinary 

2. Involves students in sustained effort over time 

3. Involves student decision-making 

4. Involves students in collaborative work 

5. Has a clear real-world connection 

6. Incorporates systematic assessment throughout the project 

7. Takes advantage of multimedia as a communication tool 

The fact that only one of the seven characteristics specifically mentions technology attests to the 
inseparability of curriculum, pedagogy, and media in the successful use of technology for learning. 
Successful technology projects need much more than good technology. 

In particular, the improved content learning in Challenge 2000 Multimedia classrooms has been 
supported in at least three ways: 

• extensive teacher professional development support 

• ongoing assessment of student work - in progress and at project-wide exhibition events 

• patience - allowing time for students and teachers to reach proficiency sufficient for high-quality 
multimedia-based learning 

How do these factors interact with teachers as they try to use technology and uphold their responsibilities 
to help students leam appropriate content and related disciplinary practices (such as historical research)? 
To find out, we'll look at the experiences of two Challenge 2000 teachers during the 1998-1999 school 
year. 



Views from the field: Two teachers' experiences 



Greta Barstow is a middle school history teacher and a teacher/leader in the Challenge 2000 Multimedia 
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Project. Oscar Jarret teaches a mixed fourth and fifth grade class and is one of the Challenge 2000 project 
teachers. Both are experienced teachers and technology users who have implemented multimedia 
projects in their classrooms and have been with the Challenge 2000 Project for several years. With a 
certain amount of technological mastery, each responded enthusiastically to the idea of improving the 
content in students' multimedia projects this year. Both decided to do this in part with greatly-increased 
formative assessment to help focus students on the content in their projects. As Ms. Barstow told her 
history class, "This is a history class, not a computer class. So I'm going to be looking for evidence that 
you learned some history in this project." 

Ms. Barstow's project was for students to develop a virtual museum on the web that would help visitors 
learn about Chinese history through Chinese art. Students worked in small groups, with each group 
focusing its work on one Chinese dynasty. They developed HyperStudio stacks (later ported to the web) 
with photographs of artwork from the dynasty, related poems and text about history, religion, and culture 
depicted in the artwork. The stacks also contained photographs of art replicas the students had made 
themselves. 

Ms. Barstow developed a series of handouts titled "Is My Project Good?" This was a question students 
kept asking her, and she wanted them to learn to answer it on their own. To do this they would critically 
look at their own work with their teacher, with peers, and alone. The handouts asked students to answer 
questions such as, "What connections did I make between art and other aspects of Chinese culture?" "Is 
the information written in my own words, in an interesting way so that my peers will enjoy reading it? If 
not, what do I need to change/add?" As the versions of "Is My Project Good" evolved, she made the 
questions more specific and scheduled more opportunities for students to assess by using the forms. For 
example, with the first form she asked each group to present their in-progress HyperStudio stacks to the 
class for comment. With the second, she conferenced with each group as they worked to answer the 
assessment questions together. Formative assessment continued throughout the project. The results were 
impressive; the content in student projects far surpassed her expectations based on work done earlier in 
the year, as well as in comparison to projects her students had produced in previous years. She was 
particularly pleased with the progress made by the class she considered to have the weakest skills. 

Still, despite her focus on content and experience in technology, Ms. Barstow experienced many setbacks 
and frustrations. She experienced: 

• Competition for resources : Many teachers schedule technology projects for the end of the year, 
and everyone needed the computer lab at once. Ms. Barstow only got one week of computer time, 
and for the rest of the project, students had to take turns using the one computer in her classroom. 

• Pressure to cover the curriculum: As is common for teachers with a large amount of curriculum to 
"cover", she fell behind as the school year drew to a close. She felt pressure to bring the project to 
completion and move on, even though many students would have benefited from more time. 

• Insufficient student research skills: Ms. Barstow was pleased to see students noticing missing 
content in their projects and begging for library time to find more. Her pleasure turned to 
disappointment when she saw that her students' research skills were often too low for them to find 
the information they needed. Although she tried to supplement skills as problems came up, time 
was too short for her to make much progress. 

• Inefficient technology use: Students tended to fall into very labor-intensive methods for getting 
their information into the computer. For example, they entered text in a way that made it almost 
impossible to edit without completely retyping it. Then, when students decided to revise, they had 
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' to spend precious computer time retyping long blocks of text. It was almost unbelievable how long 
simple changes could take - a whole class period to reorganize one screen. 



In another school, Mr. Jarret was experiencing his own set of triumphs and frustrations as his fourth and 
fifth graders worked on their Habitat project. They created an on-line guide to habitats they had visited. 
Earlier in the year, they had made huge dioramas of these habitats with paper mache animals, plants, and 
posters that told about the animals, their place in the food pyramid, and their life cycle. Now they 
photographed their dioramas, did additional research, and created web pages about each animal and 
habitat. This web-based multimedia project was one of three the class was working on concurrently. 
There was also a project about artists and another about a class project based on the work of the artist 
Hokusai. 



Mr. Jarret tried to support student learning in three areas: content, collaboration, and use of multimedia. 
These are the areas judged at the Challenge 2000 Student Interviews and defined in the Challenge 2000 
Multimedia Rubric. Mr. Jarret scheduled a series of assessment events during the project to help students 
come to a consensus on what it meant to have good content and multimedia and to collaborate well. For 
example, he scheduled two whole-class design reviews. Students used a rubric that they had developed, 
based on the Challenge 2000 Multimedia Rubric, to critique their classmates' works-in-progress. They 
asked such questions as, "Is the work organized? How might they make it even more organized?" In this 
way the class developed a sense of what it meant to have a web page be organized. They spent a lot of 
time putting themselves in the shoes of imagined web page viewers, and deciding if such a viewer would 
be able to understand the information and would want to keep looking at the site. 



The assessment events were very effective in helping students orient to the audience outside their 
classroom - once their work was on the web, anyone might see it. Several students gave this as a reason 
why they did additional research on their projects. Assessment events also helped students see how they 
could use photographs, drawings, and text together to express what they knew about their habitats and 
animals. Because of the concurrent work on collaboration skills, students were able to work efficiently 
and independently away from the teacher. Mr. Jarret gave them a small set of technology tools and 
procedures for doing the work, and once these were mastered, students used them in a fairly uniform way 
to get the work done and create the web site. They could concentrate on content and organization without 
too much attention to technology. They used Adobe PhotoShop to edit and size content elements such as 
photographs, scanned drawings and maps. They used Claris Homepage to create tables with the content 
elements they had edited in PhotoShop and descriptive text. Mr. Jarret pointed out particular tools and 
procedures in each program (e.g. setting the background color in a table cell, setting the text color and 
size, importing a JPEG picture). Students who became technology helpers used these tools and developed 
procedures for their work until they could help each student organize his or her page in a short time. Each 
student became a content expert on his or her animal, and designed and index-card based storyboard to 
prepare for making the web page. As the deadline of the Multimedia Fair approached, the class became 
an efficient working group and churned out several pages a day. All in all, the project went smoothly. 



Still, there were frustrations related to: 

• Learning the technology: This was the first time Mr. Jarret had done a web site project, even 
though he had done many HyperStudio and PowerPoint projects before. He had to leam the 
technology almost concurrently with his students. Some work had to be redone as the class came 
upon some unexpected limitations of the methods that Mr. Jarret knew to link pages together. This 
points to the difference between the level of learning a teacher gets in a training session and the 
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' complications of real technology use in a big project. 

. • Running out of time: Mr. Jarret felt that the time spent in assessment and preparatory work was 
valuable, but it meant that little time remained to get the sites done before the district Multimedia 
Fair. In the end, a few parent volunteers spent a long night before the Fair checking and linking 
pages. 

• Keeping it too simple? Mr. Jarret felt that he had to limit what students put on their web pages to 
the kinds of media he knew how to use. There was little time for experimentation. Therefore 
nobody used sound or animation. Pages all were built the same way - as tables. Mr. Jarret felt that 
this was unavoidable if the project was ever going to get done. 



Learning from classroom experience 

These classrooms seem like some of the busiest places on earth. A computer crashes and students 
explode with frustration. A group gets an animation working - everyone crowds around to see. There's a 
line in front of the scanner. Kids leaf through books looking for just the right picture. The teachers 
alternate between resetting the printers, looking for more paper, calling the library to see if students can 
get in, and helping students understand content. 

We can see why it is so important for teachers to be able to network with other teachers. So many 
different problems come up during a project that there is no way a teacher can completely prepare in 
advance. Teachers need sources of just-in-time advice. Each emergent problem, once solved, becomes a 
tidbit of knowledge that might just save another teacher a few days of frustration. 

The two teachers' stories show that even fairly experienced teachers struggle to balance content 
integration with available time and resources. Even experienced teachers have to leam new technology, 
because it is always changing. They also have to find ways to make a technology project teach enough in 
terms of the subject matter to make it worth the time it takes. 

Yet, like the middle school math teachers, both of these teachers can hardly wait for the next year, so 
they can put to work all the insights and skills they acquired during these projects. Ms. Barstow wants to 
spend more time conferencing with groups. Mr. Jarret wants to develop templates for students so that 
they can start their projects at a higher level of technology use. Both were excited about the way their 
students learned content, and also learned to manage time, collaborate, design, and use new software. 



Capitalizing on the Tension of Integrating Technology 



The dilemma of learning both computer technology and content exists and will continue to persist. There 
will always be new computer technologies to leam and there will always be new ways to approach the 
learning of content. In fact, the problem is a wonderful paradox because technologies have made it 
possible for many teachers to see that complex ideas and abstractions-the parts of content learning that 
seem so difficult for students to accomplish-are actually made more accessible through the use of 
computer technologies. Teachers are seeing that classroom content can be more than assembling pieces 
of knowledge to be learned, and that technology can offer representations, visualizations, and interactions 
that really help students negotiate concepts and abstractions. Conceptions of what should be taught and 
how it can be taught are now in flux, and computer technologies are playing a role in demonstrating how 
subject area standards can be realistic and accessible for students. Teaching is complicated and computer 
technologies, like other technologies that came before, create affordances and constraints in the learning 



O 



152 

www.ed.gov/TQChnology/TechConf/1999/whitepapers/paper4.html (8 of 10) [4/24/2001 2:38:20 PM] 



008003061600 



The Technology/Content Dilemma 

process. In both MMAP and the Challenge 2000 Multimedia Project, teachers are working to reconceive 
their approaches to content as well as their approaches to the media, tools and classroom and virtual 
activity structures. As always, this process is at the heart of teaching. Teachers from both projects have 
explained how integrating technology into their classrooms has brought a revitalization to their teaching. 
They are no longer using the same materials year after year, and they feel that they are getting to learn 
alongside their students. 

The technology leaming/content learning dilemma necessitates a call for more complex models and 
experiences for teacher professional development and more materials that support standards-based 
learning. Our work at IRL has centered around creating materials and helping teachers create formal and 
informal opportunities for networks and communities in which to learn technologies and to work on 
these teaching dilemmas. We advocate for teachers to have time to experiment with technologies, share 
best and worst practices, study exemplars of student work, and deal with conflicts, successes and 
disappointments in their attempts with computer technologies. Once teachers have engaged with 
technology and have seen students engage, shine and go beyond their expectations, they are willing to 
cope with the tension between attention to technology and attention to content. They need to carve out 
time and become proficient at being in a classroom that feels like the busiest place on earth while staying 
focussed on pedagogy. It's a tall order, but we are seeing more and more teachers succeeding. 
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Does educational technology work? Does its application in classrooms help children learn? Can teachers 
improve their own understanding and practice through technology? Y es! Yes! ! And yes! ! ! 

In a period of widespread concern about educational quality, teachers, parents, policymakers, and 
taxpayers deserve answers that go beyond fervent beliefs and jaunty assertions. They need evidence in 
order to calm their doubts, justify their expenditures, and strengthen their confidence in what we do. 
Because we have developed the most sophisticated evaluation methods in the world, we should be able to 
document strengths and identify shortfalls of technology-based learning systems. 

Presented here are a brief set of ideas and guidelines for you to consider, ending with how technology 
itself can aid in the testing and evaluation process. Let's start with the core notion that evaluation should 
be planned at the beginning of an innovation rather than tacked on at its end. Evaluation is a planning 
tool as well as a way to systematically collect and interpret findings and document impact. Scholars 
(Baker &Alkin, 1973, Scriven, 1967) have divided evaluation into two types: formative evaluation- where 
information focuses on program improvement; and summative evaluation-where information is used to 
make a decision among options or to certify the effectiveness of a program. In reality, all evaluation is 
now both summative and formative: Data help designers and users to improve practice (because nothing 
works right the first or second time) and also give information about whether the innovation is 
sufficiently promising to continue investing in. Technical standards for the conduct of evaluation have 
been produced (AERA, APA, & NCME, Standards for Educational and Psychological Testing , 1985). 
The degree to which evaluations demand many of these concerns depends on where the innovation is 
going and who has to be convinced. Who are the main consumers of information-the teachers and 
students in the innovation, its funders, policymakers? Does the evaluation need the blessing of an 
external evaluator or consultant, to give an aim's length picture of the process, or are you comfortable 
about building and improving your own systems? Decisions on this score may help you decide whether 
TO solicit external help or do it yourself. If the former, the guidelines may help you design the kind of 
request for proposal you want and the kind of standards for work you will accept from a subcontractor. In 
either case, the ideas below are intended to help you think systematically about what you're doing and 
how to capture and document accomplishments. 

Technology for What? 

What is the technology intended to do? Tom Glennan distinguishes between technology-pull and 

155 



o 



wed.gov/Technology/TechConf/1999/whitepapers/paper5.html (1 of 5) [4/24/2001 2:38:25 PM] 



Technology: How Do We Know It Works? 



100000001603 



technology-push (Glennan & Melmed, 1996). Goals for classroom technology can focus on learning the 
use of tools to assist in other areas of leaming-for instance, using search engines, e-mail, databases, 
spreadsheets and word processing to find, analyze, represent, and produce documents and other products 
to display learning. This type of learning may be related to standards set for the school or the state's 
children to meet. The focus is on using technology to meet requirements. The requirements pull the 
technology to them. 



A second set of goals may be to use technology power to address new goals that cannot be met in any 
other way. These could involve the designing of complex simulations, or the collaborative interaction on 
projects with scientists, other experts, and other students across the nation and the globe. In this case, the 
technology itself pushes users to new goals and new options. 



The object of a third set of goals is to use technology more efficiently to deliver instructional 
opportunities that match the background and pace of the learners. Such uses typically involve integrated 
programs where students are helped to acquire specific knowledge and skills. 

There are also technologies that focus on the management of classrooms by teachers, but for the moment, 
let us address the evaluation of students' learning. It is absolutely critical for evaluation to determine the 
degree of emphasis among the three kinds of goals identified above, to be clear about them, to 
communicate them to all collaborators, including students, and if revisions occur, to be open about how 
goals have changed. 



Technology Innovations: The How 

In every evaluation we have conducted, the road is rocky at the beginning. And whether the evaluation is 
well funded or operating on a shoestring, hardware and software may not arrive when expected, 
infrastructure may be delayed or wrong and in need of adjustment, technical assistance may be not 
fashioned exactly to meet the users' emerging needs. So expect this small amount of chaos. 

A good evaluation considers, in addition to goals, who the key participants are-administrators, teachers, 
parents, students, software providers, consultants-and whose roles are key at what points. Remember also 
to note how decisions are made to adjust the program, whether they are explicit, and how to keep track of 
them. This part of evaluation is just good planning. 

Implementation of an innovation also depends on a lot of different factors. First, perhaps, is the locus of 
the ideas for the work. Is it a school-based innovation led by teachers? Is it a collaborative venture 
involving software that needs to be customized and integrated into a curriculum for particular students or 
regions? Is it an externally imposed "opportunity" depending upon volunteers or incentives? How 
systematic is the use of the innovation over what time period? Are we talking about a neat activity that 
takes a week, or a long-term set of skills (such as modeling and representing data) that can be useful over 
the long haul and in which it takes a substantial time to develop expertise? Is the project one that 
emphasizes motivation? The excitement of communication with other students rather than the 
development of content expertise? 

How much documentation about implementation is needed? The schedule and timeline of the beginning 
and key junctures in the innovation? The integration (or lack thereof) with regular parts of the 
curriculum? Training requirements and systems for teachers, students, and other participants? 
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Are the. learning topics intended to include the full range of the curriculum? To focus on certain subjects, 
for - instance, history? To concentrate on one or two topics within courses, like earthquakes in an earth 
sciences course? Is the emphasis interdisciplinary? Is the topic a matter of student choice, and if so, how 
is activity linked to important expectations? 



What is the scope of the project? A few teachers at one school? Teams of teachers at the same grade in a 
part or all of a district? A statewide scale-up of computer-based curricula? Foundation-supported 
innovations of different characters and goals at different sites? 



Who Benefits? 

Which children or students (and teachers) are the key beneficiaries of the innovation? Is there specific 
background learning or experience that makes children particularly ready for the innovation 
planned-including language,. computer skills or lack thereof, out-of-school experiences, content 
knowledge? Are the children located at a particular age range or grade level? Are they supposed to be 
affected over a number of days, weeks, or years? What is a fair comparison group? Others in the school? 
Children at other schools or sites? 



Other Evaluation Considerations 

An innovation also has a set of philosophic underpinnings that might need to be considered. Is emphasis 
placed on exploration and collaboration? On mastery and fluency? On subject matter depth or 
generalization to a number of topics and subjects? Each of these potential emphases, and many others, of 
course, may need to be evaluated. 



Measures of Outcome and Impact 

A few words of advice. Don't hinge the evaluation findings on who likes what. Teachers' descriptions of 
their "excitement" and students' enthusiasm are certainly desirable, but are probably unlikely to persuade 
external decision makers of the success of an innovation by itself. If that enthusiasm links to fewer 
absences, or more attentiveness, then the evaluation will gain power. As an overall dictum, focus first, 
intensely, and last on student learning. Such a concentration will refer you back to your original goals 
and may require a redefinition of your original intentions. 

Measuring outcomes involves two main components: what you will use to provide the data, and how you 
will decide whether the findings are sufficiently good to warrant continuation, revision, and so on. 

Types of measures include regularly administered tests, either commercial or statewide assessments. 
There may be special tests already available to measure students' acquisition of the particular area of 
focus. Often the tests and measures may need to be developed to tap into new uses to which the computer 
is put. These other measures may include projects, essays, and extended performances, as well as typical 
tests of knowledge and skills. You need to be sensitive to the fact that if you use open-ended tasks such 
as performance or essay examinations, you need to use clear criteria to judge performance, and 
performance should be validly and consistently measured among raters. You should remove, to the extent 
you can, the bias inherent in having teachers rate their own students or the performance of only students 
known to be in the technology option. Questionnaires asking about student attitude, ease of use of the 
applications, and suggestions for improvement from those who participated in the technology may also 
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be helpful. 

The-most frequent way that evaluators determine whether performance is good enough is by using 
comparisons. You can compare students in and out of the innovation (although to be certain, you should 
assign them randomly rather than just using intact classroom groups). You can use pretest versus posttest 
scores, particularly if you have comparison groups of similar students. If you use pretests and posttests, 
you'll probably need some external help to deal with the practice effects of the test (learning from the 
test), interaction effects (how the pretest may enhance the impact of the technology), and the reliability of 
the measure you use (the difference between pre and post, or a more sophisticated statistical analysis). 

All this help is readily available. You may also want to follow up students and look at their performance 
over time, even after they are through with the particular program of interest, in order to determine 
whether there are long-term effects. When you have sufficient numbers, you should disaggregate your 
results to see whether the innovation works better for students with certain backgrounds, particular 
experiences, or specific knowledge. 

The major trade-off is whether you link results to the regular test (policymakers would like that) and 
recognize that it is generally much harder to show impact in this way than on assessments targeted 
toward the same content and cognitive demands as the innovation. Your local policies may be your best 
guide here. 

Technological Supports for Evaluation 

It makes most sense, of course, to use measures that optimize detection of impact for the innovation you 
are developing. For that reason, we advocate the use of computer-based assessments where possible and 
where they have sufficient technical quality, including validity and reliability evidence. CRESST has 
developed measures of problem solving, content understanding, knowledge representation, search 
strategies, collaboration, and Internet learning, for example, that can be administered by computers. 
Ideally, you would want to automate information about how students are engaging in their technology 
use to help you understand why you have obtained given results. Maybe students do best who have a 
slowly increasing involvement. Maybe there is a threshold that allows them to take off. Maybe their lack 
of background content knowledge is holding them back. 

A second kind of support that CRESST has is a database manager (called the Quality School Portfolio, or 
QSP) that allows the user to transform databases (for instance, of district or state scores) into a local, 
longitudinal database for all students. Then students in the technology innovation and those in the 
comparison group can be sampled on various bases-background, prior subject matter grades, test 
scores-and the data disaggregated immediately. QSP also allows the use of locally developed outcome 
and attitudinal measures by providing a resource kit of measures, guidelines for their use, and scanning 
and analytical capability. In the end, QSP generates a report comparing groups, or a single group at 
multiple time points. Graphical reporting can be tailored to various audiences for the report. 

Summary 

To sustain and support the growth of high-quality technology in schools, everyone has to learn to be 
more aware of what standards of documentation are useful. Each of us can learn to interpret quality 
information to revise, redesign, or reconceive the ways technology can be used to help our children meet 
our expectations. Better that we have a hand in it. 

158 

rV - vww.ed.gov/Technology/TechConf/1999/whitepapers/paper5.html (4 of 5) [4/24/2001 2:38:25 PM] 



Technology: How Do We Know It Works? 



References 



American Educational Research Association, American Psychological Association, National Council on 
Measurement in Education (1985). Standards for educational and psychological testing. Washington, 
DC: American Psychological Association. 

Baker, E. L., & Alkin, M. C. (1973). Formative evaluation of instructional development. A V 
Communication Review , 27(4), 389-418. (ERIC Document Reproduction Service No. EJ 091 462) 

Glennan, T. K., & Melmed, A. (1996). Fostering the use of educational technology: Elements of a 
national survey (MR-682-OSTP/ED). Santa Monica, CA: RAND. 



Scriven, M. (1967). The methodology of evaluation. In R. E. Stake and others (Eds.), Perspectives on 
curriculum evaluation. AERA Monograph Series on Curriculum Evaluation, No. 1. Chicago: Rand 
McNally. 

4LU -U 



fThe Technology/Content Dilemma! 
Technology: A Fly-Over of Policy Questions! 





-< 

PREV | 







[Documenting the Effects of Instructional 



159 



o 



w.ed.govrTechnology/TechConf/1999/whitepapers/paper5.html (5 of 5) [4/24/2001 2:38:25 PM] 




Documenting the Effects of Instructional Technology 

The Secretary's Conference on Educational Techn ology-1999 

Documenting the Effects of Instructional 
Technology: A Fly-Over of Policy 

Questions 

by 

Dale Mann, Professor Teachers College, Columbia University 
New York, Ne ’ York 10027-6696 
and 

Managing Director Interactive, Inc. 

Huntington, New York 11743-3360 

PREFACE 

I. EVALUATION RESEARCH ABOUT INSTRUCTIONAL TECHNOLOGY POLICIES 

II. PEDAGOGY 

. The Efficacy of Instructional Technology 

B. Pedagogy: The Multiple Sources of Learning 

C. Pedagogy: Learning Outside the School 

D. Implications for a Research Agenda 

1. IT Effects @ Home 

2. IT Effects @ School 

3. The Effects of Serious Play 

4. Estimating the Critical Mass of IT 

5. IT Dosage 

III. POLITICS 

IV. ECONOMICS 

. Capitol Decisions and Capital Decisions 

B. The Public Benefits of Private Investment 

C. The Roles of Government 

V. FOUR INSTRUCTIONAL TECHNOLOGY ADDITIONS TO THE EXISTING CONCE PTION 
OF SCHOOLING 

. Adding an Education Focus to Our School Focus 

150 

O AV.ed.gov/Technology/TechConf/1999/whitepapers/paper6.html (1 of 9) [4/24/2001 2:38:30 PM] 

:RIC 



Documenting the Effects of Instructional Technology 



000000001608 



, B. Adding a Learning Focus to Our Teaching Focus 

% 

C. Adding Homes to Schools: Adding Parents to Teachers 

D. Moving Learning to the Learner 

VI. FOUR EVALUATION RESEARCH QUESTIONS FROM FOUR PERSPECTIVES 



PREFACE 



[October 22, 1707, The English Channel] 

"Returning home victorious from Gibraltar... [Admiral] Sir Clowdisley... summoned all his 
navigators... The consensus placed the English fleet safely west of... the Brittany peninsula. 

But as the sailors continued north, they discovered to their horror that they had misgauged 
their longitude near the Sicily Isles [which] became unmarked tombstones for two thousand 
of Sir Clowdisley's troops... [and] four of the five warships."! 

Appalled by the loss of lives and ships, in the Longitude Act of 1714, the Parliament promised a prize of 
£20,000 for a better way to navigate than throwing a log overboard and watching it drift off. Without a 
way to measure time accurately, ships could not determine noon and without a way to determine noon, 
they could not determine their east-west location. The lack of measurement had serious consequences. 

1 begin with two assumptions: first, that we want to do credible science and second that we want that 
science to advance the contribution that instructional technology might make to learning. Trying to 
advance instructional technology makes my perspective partisan or political in the sense that I am 
acknowledging a particular value interest in the outcomes of this application of science. 



I. EVALUATION RESEARCH ABOUT INSTRUCTIONAL TECHNOLOGY 
POLICIES 

"640K ought to be enough for anyone." Bill Gates, 1981 

Doing research costs money and most evaluation research is paid for by clients — governments or private 
organizations. Adding a client may complicate evaluation in the same way that adding a patron 
complicates art: some clients and some patrons have expectations in addition to the outcomes of the 
otherwise pure event of science or art. The evaluation fish swims in the sea of politics, and should. Anne 
L. Bryant, executive director of the National School Boards Association says, "School Boards are going 
to be asking increasingly: 'Demonstrate to us that [computer-based instruction] has results."^ 



Imagine a senior government official on the cell phone, in a cab, running late on her way up Constitution 
Avenue to a congressional hearing. She knows she will face pointed questions about "All these 
computers we put in schools". If you answer the call, will you want to help address that skepticism? If 
you are good at helping, you might add seven figures to the appropriations authorization: if you are really 
good, you might add eight figures. If you want to help, you have to compel belief and that is likely to be 
more than good science, it is likely to address pedagogy, politics and economics, all at once. 



In my view, evaluation research about education policy is intended to effect decisions and typically 
addresses pedagogy, politics and economics, all simultaneously. 
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II. PEDAGOGY 

"There is no reason why anyone would want a computer in their home." Ken Olson, 

President, Chair and Founder of Digital Equipment Corp., 1977 

A. The Efficacy of Instructional Technology. The overriding question is how powerful is instructional 
technology? A second order question is, how do we know how powerful that technology is? 

First, I believe that instructional technology works.^ Instructional technology only works for some kids, 
in some topics and under some conditions but that is true of all pedagogy, all systems for teaching or 
learning. There is nothing that works for every purpose, for every learner and all the time. 

Emphasizing the things that instructional technology has not done has its political uses just as surely as 
saying that technology works. One continuity among critics of instructional technology is the idea that all 
teachers are always preferable to all machines, e.g., William L. Rukeyser's statement that, "The best 
teacher has always been a person, not a machine, "d The sub-title of the cover story of The School 
Administrator for April 1999 was "A Leading psychologist calls for slowing the rush toward computing". 
In that piece, the critic of instructional technology, Jane Healy acknowledges that, "...(W)ell 
implemented simulations and conceptually driven programs may improve learning — if a good teacher is 

in charge. 

But what is known about the learning efficacy of such ubiquitous features of American schooling as the 
teacher-talk model of instruction? The 770 square foot classroom box? The 180-day (American) school 
year? We accept and even welcome critical attention to instructional technology that is seldom applied to 
the implacable regularities of American schooling. That leads to a paradox in which technology from the 
last generation has been proven inadequate and that from the next generation is unproven. With either 
negative data or none, the field is left to those who promptly make the next generation of technology the 
worst enemy of the current generation as in, 'Next year it will be cheaper, faster, smaller or even— more 
constructivist. So let's wait.' 

Our goal should be first, to understand the conditions of pro-social technology use and second to employ 
that understanding for learning improvement. Both require more penetrating analysis than has heretofore 
been the standard. 

B. Pedagogy: The Multiple Sources of Learning. 

The first thing to be understood is that there are many, many sources of learning. Technology needs to be 
disentangled from the other sources. Some are inside and many are outside the school. Parents educate, 
the family educates, the media educates and so on. Children learn from their teachers, from textbooks, 
from homework, from the Channel One TV on the wall and they learn from computers.^ 

C. Pedagogy: Learning Outside the School Thus, the contribution of instructional technology is best 
understood in a context that includes the contribution of all the educators. 

Since James S. Coleman's 1966 analysis, it has been generally acknowledged that about 30% of the 
variation in children's educational achievement comes from their experience in school and 70% comes 
from other experiences, especially their families, the culture they live in (the media, etc.) and their peers. 
Coleman's insight works both ways. Families that support learning, advance their children's educational 
achievement: those that do not or that hinder or disrupt learning, impede their children's educational 
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achievement. In later writing, Coleman called the leverage that families apply to their children's learning, 
"family capital". 

D. Implications for a Research Agenda. There are several implications for an agenda of evaluation 
research about instructional technology. 

1. IT Effects @ Home. We need to account for all the educators — school effects and home effects. How 
many studies are there of the amount of learning at home that is supported by instructional technology? 

2. IT Effects @ School. Inside the school, we need to find ways to measure technology effects separate 
from teacher or textbook effects. How many studies attempt to measure the amounts of these various 
phenomena and associate them with outcomes? 

3. The Effects of Serious Play. Except for the 'learning-should-hurt' crowd, most educators recall what 
coaches and early childhood educators have never forgotten — play is a child's work. Entertainment is 
correctly pilloried as passive and generally purposeless. But active play is strongly connected to learning 
of all sorts.? The fact that we do not know much empirically about what and how children are learning 
from technology-delivered serious play applications outlines the limits of our imagination. 

4. Estimating the Critical Mass of IT. We do not have any very good way to answer school board 
questions about, How much is too little? Few school boards would accept a coach assertion that 42 
minutes a week of basketball practice, in groups of 24, will result in 42 offers of college athletic 
scholarships. Why then do we allow policy makers to believe that 42 minutes a week of "Computer" 
from the "The Computer Teacher" in "The Computer Center" will change children's school performance? 

5. IT Dosage. Most current evaluations assume that if a school has bought a site license for Electrified 
Reading: Release 2.0 then the teachers are using it and the children are experiencing (some unknown 
amount) of it. A generation of so-called "implementation research" suggests the fallacy of that 
assumption yet we do not have good measures of children's exposure or even of teachers use of 
programs. We need much more attention to elapsed time, exposure effects, dosage effects. 

III. POLITICS 

"The wireless music box has no imaginable commercial value. Who would pay for a 
message sent to nobody in particular?" Advice to David Samoff in the 1920's. 

Politics is the process through which values are authoritatively distributed for a society. Whatever its 
interpretation in the popular culture, politics has deep implications for the purposes served by 
government action, for example, which children get what quality of schooling and which not? If 
measurement is the essence of science, benefit is the essence of public service, analysis may document 
who benefits from a particular program; politics determines how that benefit will be distributed. 

Consider a choice among three public policy options: What would advance the children's interests more: 

. Higher pay for already employed teachers? 

B. The same amount of money but spent only to hire additional teachers? Or, 

C. The same amount of money spent on instructional technology? 

Or, assume that the policy goal is to "integrate technology into the classroom". The instant consensus is 
that can be done only by more professional development for teachers — more in-service, more released 

163 

vww.ed.gov/Technology/TechConf/1999/whitopapers/paper6.html (4 of 9) [4/24/2001 2:38:30 PM] 



O 



000000001611 " 

Documenting the Effects of Instructional Technology 

time,- more contracts to Teacher Centers or, my own favorite, more subsidies for graduate school tuition. 
We never consider options such as: 

. Giving teachers a computer to take home over the summer (and trusting their professionalism, 
curiosity and commitment) 

B. Putting computers into the classroom and letting the kids explore them and co-teach, co-leam with 
their teachers. 

C. Using technology to teach technology — for example, by producing CD-ROM role-playing 
simulations about what happens in classrooms under different conditions and with different 
teacher choice-consequence paths. 

Neither do we consider making technology so 'transparent' that it does not require training, for example, 
ATMs. For the most part, policy choices are limited by political power and by the conventional 
conceptions of education as schooling and of learning as teaching. They all lead to the same 
labor-intensive conclusions and all are centered on teachers not on learning. We do not, for example ask 
the following question: 

Under what conditions are protein-based teaching systems preferable to digital learning systems? 

It is at least possible that digital systems do things that RLHB's 

• should not bother to do (keep records) 

• do not want to do (drill children) 

• can not do (have infinite patience) or 

• do not do reliably (treat all children as though they can all learn)? 

Or consider "The Learning Odyssey", a complete curriculum for grades 4 though 9, produced by the 
Agency for Instructional Technology (AIT) that was originally aimed at the home-schooling market. The 
topics include language arts, math, science, history, art, music, technology and personal development. All 
subjects are aligned to state content standards. Teacher comments on student work are available by 
e-mail. Subscription prices are $1 50/month; $350/3 months; $900/9 months; and, $1, 100/year. As part of 
the price: AIT will pay for a child to be tested with any standardized test required by a local jurisdiction. 

AIT describes the "Learning Odyssey" as a replacement for school. "..(L)eaming need not be 
school-based. ..schools must reinvent themselves as institutions with a far greater purpose, or cease to 
exist. "& 

Or consider another volatile issue, violence and the Internet. Post-Littleton, the concern to minimize the 
sources of violence in children's lives is likely also to generate unintended negative consequences. Our 
understandable attention to the harmful examples of Internet use may cripple wholesome applications of 
the same technology 

We can help children by minimizing violence but also by maximizing good. If we would not ban all 
pharmaceuticals because some are hallucinogens, then we should also differentiate between pro- and 
anti-social applications of telecommunications. Except for a few one-off examples of good video games 
(Tetris, Carmen Sandiego), we have no systematic understanding of the good that can be done through 
learning related games and the Internet. 

In order to encourage more wholesome development of these technologies, we need to understand how 
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they work. To continue the pharmaceutical analogy, we need to identify the active ingredients in these 
applications followed by clinical trials to document their effects. We need clinical trials that (1) identify 
and measure the active ingredients of instructional technology and (2) that document the gains associated 
with amounts of their use. 

Whatever the case, analysts, researchers and/or evaluators do not have the right to make deeply political, 
deeply value choices. Analysts are not elected or authorized by any constituency to make official 
decisions. Doctorates are not licenses to usurp state legislatures, local boards of education or even 
superintendents and principals. The role of analysis is to inform decisions, NOT to make them. 



IV. ECONOMICS 

"I think there is a world market for maybe five computers. " 

Thomas Watson, Chairman, IBM, 1943 

A third of a century ago, James Coleman and Lawrence A. Cremin tried to teach us that we needed a 
more generous vision of education than one centered exclusively on schools. In addition, we need a more 
generous vision of progress than that which depends solely on public funds. There are partners who share 
important and child-centered purposes and they are in the private sector, perhaps especially in technology 
in the private sector. 

A. Capitol Decisions and Capital Decisions. Public and private decision-makers are interested in 
answering the same question: What works? They are because both are making investment decisions. "We 
need information to show what works and what doesn't. If we had empirical data, policy-makers would 
be more willing to fund technology and voters would be much more willing to pay."2 

The interest in 'what works' goes beyond public policy. There is now a category of inquiry called 
"curriculum due diligence". Because potential investors have a right to know about the integrity of what 
is being offered, banks and brokers retain curriculum analysts to document those companies in the 
learning business can actually deliver what they claim to sell — learning. Whether the goal is benefit to 
the constituents or return on investment, the interest in efficacy is the same. 



And in both instances there are competing uses for the same funds. Linda Roberts, the director of the 
Office of Educational Technology in the US Department of Education said, "School districts will be 
called to task for 'What are you doing with your money and what difference does it make? "12 In the 
public sector, the (implicit) questions are: More highways or more schools? More lights on police cars or 
more computers in classrooms? More scholarships for college students or more professional development 
for teachers? The private sector compares buying a magazine to creating software, or starting a chain of 
day care centers to creating an Internet homework helper site. All of those decisions can be illuminated 
by data about outcomes for learning. 

B. The Public Benefits of Private Investment. Where has there been more good for schooling? (A)The 
clouds of quarter million dollar grants from foundations and State Departments of Education to fund 
(non-profit) pilot projects and experimental activities or (B) two guys named Jobs and Wozniak trying to 
breadboard a "home computer" in a Cupertino garage? 



We should at least admit that in a capitalist society the engine of innovation — and yes, largely of 
improvement — is the profit motive. The need for parents to be better connected to their children's 
learning is widely acknowledged but who has invested more in creating a curriculum of the home? State 
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departments of education or the Lightspan Partnership? (Hint: Creating the 100+ CD ROMs that support 
reading, language arts and mathematics, grades k through 6 has cost Lightspan $150 million. 

C. The Roles of Government. Recall that the Internet started out as a Defense Agency Research Projects 
Administration experiment to connect weapons labs. The Food and Drug Administration supervises 
clinical trials of privately developed pharmaceuticals on behalf of the public. 

V. FOUR INSTRUCTIONAL TECHNOLOGY ADDITIONS TO THE 
EXISTING CONCEPTION OF SCHOOLING 

"Computers in the future may weigh no more than 1.5 tons. ""Popular Mechanics", 1949 

A Adding an Education Focus to Our School Focus. There are a lot of educators in every society — the 
TV, newspapers, parents, religious and cultural institutions, video games, sports and the general culture. 
Every time we hear "education" and automatically think "school" we are diminishing the prospects of 
improvement. 

B. Adding a Learning Focus to Our Teaching Focus. The frontal act of instruction, the uncertain 
business of trying to require children to leam particular things is very difficult (ask any teacher). What if 
we re-conceptualized "the teaching of children" as "the facilitation of learning"? Likely, it is more 
possible to arrange learning than it has been to force teaching. 

C. Adding Homes to Schools: adding Parents to Teachers. Schools and homes remain isolated from each 
other. And, despite there centrality in the lives of children, we have never had very good bridges between 
the two. Using Lightspan's Achieve Now! Schools lend children a Sony Playstation (retail cost, $100) as 
a platform for learning-related video games that are launched by the teacher in the classroom but then 
completed by children and their parents at home. In a pre/post and experiemental/control evaluation, the 
children and schools with this home-school-home connection performed better on reading, language arts 
and mathematics than did those without. 

Moving Learning to the Learner. One certain consequence of digital technology is that learning will go 
to the learner. In the earliest times, boys went with their fathers and uncles to observe the hunt; girls went 
with the mothers and aunts to discover which plants were edible. The artists of the cave walls moved 
learning inside. The creation of the common school still required learners to go to the site of learning and 
to dependent on the knowledge masters. Dependency makes learners vulnerable to the political (and 
ethnic and class and gender) prejudices of the masters. 

Digital communications reverses that commerce (with the Internet, learning goes to the learner) and 
dramatically transforms that politics. Digital learning can be "The 4 'Anys 1 — Any Learning, Any lime, 
Any Place to Any One". The democratizing impacts of that reversal are heartening but only dimly 
perceived. And the consequences for schools and universities, conceived as physical spaces, have not 
begun to be imagined although their consequences are probably captured by the observation about 
technology as a 'train' — you will either be on it or under it. 

VI. FOUR EVALUATION RESEARCH QUESTIONS FROM FOUR 
PERSPECTIVES 

"But what is it good for?" Comment on the microchip from IBM Advanced Computing 
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• systems Division, 1 968 

• From the perspective of science, How certain, how unambiguous, how compelling are the data? 

• From the perspective of pedagogy, What implications for practice can be derived from the 
evaluation? 

• From the perspective of politics, What political values are served? 

• From the perspective of economics, What are the cost or financial implications? 

About the Author: Dale Mann is a professor in the Program in Educational Administration at Teachers 
College, Columbia University and Managing Director of Interactive, Inc., a technology development and 
evaluation firm Huntington, New York. 
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1. Dava Sobel, Longitude: The True Story of a Lone Genius IVho Solved the Greatest Scientific Problem of His Time, New 
York, Walker & Co., 1995, pp 1 1-13. Sobel continues, "[the night before, the Admiral had been approached by a 
sailor], ..who claimed to have kept his own reckoning of the fleet’s location during the whole cloudy passage. Such 
subversive navigation by an inferior was forbidden in the Royal Navy. ..[the Admiral] had the man hanged for mutiny on 
the spot." 

2. Pamela Mendel, "Issues That Defined the Year for Schools and Computers", The New York Times, Wednesday, 
December 30, 1998. 

3. For some examples, see Charol Shakeshaft's paper also submitted for The Secretary's Conference on Educational 
Technology and attached herein. For a recent summary of 264 research reviews and reports, see Jay Seven-Kachala and 
Ellen R. Bialo, 1999 Research Report on the Effectiveness of Technology in Schools, 6^’ Edition, Software & Information 
Industry Association, Washington, D. C., 1999. 

4. William L. Rukeyser, "Computers' Role in Education", the website of "Learning in the real world", Woodland, CA,. 

5. Jane M. Healy, "The Mad Dash to Compute", The School Administrator, v. 4, n. 56, pp. 6-10. C. F., J.M. Healy, Failure 
to Connect: How Computers Affect Our Children's Minds - for Better and Worse, New York, Simon & Schuster, 1998. 

6. Many school people have adopted a constructivist interpretation of the work of children while continuing to focus on 
teachers as the sole source supply of children's learning. As technology becomes more sophisticated (for example, through 
artificial intelligence), this singular fealty to the school's employees will be harder to maintain. For example, the 
experience of the Internet suggests the powerful extent to which technology itself teaches with or without the mediation of 
adults. 

7. C.F., Dale Mann, "Serious Play", Teacher College Record, v. 79, n. 3, Spring 1996, pp. 446-469. 

8. Michael Sullivan, Executive Director, AIT, "The Heller Report: Internet Strategies for Education Markets", v. 4, n. 2, 
August 1998, p. 4. 

9. Lieutenant Governor Kim Robak, Nebraska, Quoted in Kerry White, "A Matter of Policy," Education Week, November 

10. 1997, v. XVII, n.l 1, "Technology Counts: Schools and Reform in the Information Age," p.6. 

10. Pamela Mandels, "U.S. Official Calls for Studies of Technology in the Classroom", New York Times, April 27, 1998, p. 
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The Secretary's Conference on Educational Technology-1999 

The Cyberspace Regionalization Project: 

Simultaneously Bridging the Digital and Racial Divide 

I. Introduction 

II. Description of the Project 

III. Theory of the Project 

IV. Measuring Racial Attitudes: The Evaluation Piece 

V. Baseline Results and Research Agenda 

VI. Conclusions 

I. INTRODUCTION 

As this century nears its end, we are a decade into the resegregation of our nation's schools. ..It has been 45 
years since Brown v. Board of Education outlawed intentional segregation in the south, but a series of Supreme 
Court decisions in the 1990's helped push the country away from Brown's celebrated ideals and closer to the 
old idea of "separate but equal. "i 

As many of the major school districts throughout the country have recently ended or phased out their 
desegregation plans^, even some of the most ardent supporters of desegregation have conceded, preferring (in 
the words of Brown University's Michael Alves) to "make Plessy work"- alluding to the 1896 U.S. Supreme 
Court decision that allowed "separate but equal" public facilities - by waging a piecemeal attack on educational 
inequity .2 Where high-poverty schools are failing, they are given extra money in compensatory funding. 

Where the curriculum is weak, standards are raised. And if teachers in such schools are underqualified, 
professional development is enhanced. 

Though in totality these individual initiatives are not going to make up for the inequities that are a natural 
result of resegregation, each is worthy of support irrespective of whether or not you are in favor of "making 
Plessy work." They are worthy of support because they will help improve traditional educational outcomes 
such as student achievement. But, what about the non-traditional outcomes? What about the intangible benefits 
of interracial contact; "those qualities which are incapable of measurement but which make for greatness in 
a... school"^ Weren't these intangible benefits of interracial contact the very essence of the Brown decision and 
its precedents? 

If we accept, for now, that resegregation is a present reality, and that Caucasian students are going to be in 
different schools than minority students, is there a way to generate the kind of interracial contact that creates 
the sort of intangible benefits the Supreme Court believed would emanate from desegregation? Digital 
telecommunications may be one means to that end. Consider the Cyberspace Regionalization Project. 

II. DESCRIPTION OF THE PROJECT 

The Cyberspace Regionalization Project uses advanced audio-visual telecommunications to bridge gaps of 
geography (70 miles) and socioeconomics between two New Jersey high schools, one white and affluent and 
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the othen black and low income. Table 1 displays some of those gaps. 
TABLE 1. 





ASBURY PARK H.S. 
(Monmouth County) 


HUNTERDON CENTRAL 
REGIONAL H.S. 

(Hunterdon County) 


ENROLLMENT 


757 


2005 


CAUCASIAN STUDENTS (%) 


4 


95 


STUDENT MOBILITY (%) 


35 


8 


DROPOUT RATE (%) 


10 

* 


2 


POST-SECONDARY STUDY 
(all types) (%) 


65 


91 


PER-PUPIL EXPENDITURE 


$9,293 


$11,633 


HPST PASS RATES (%): 






All areas 


41 


91 


Reading 


59 


94 


Mathematics 


76 


98 


Writing 


56 


97 



Using audio-visual links provided by Intel ProShare software and equipment, students and teachers from the 
two high schools work together on a variety of curricular and co-curricular activities. Much like corporate 
executives conducting a video tele-conference, real-time images of the students are displayed on a computer 
monitor while they work together on various projects such as a science experiment or an electronic literary 
magazine. Teachers, trained under a grant from AT &T— design the interactions and supervise the students 
throughout the project. 

III. THEORY OF THE PROJECT 

While the unfortunate consequences of the "digital divide" between races and social classes are often 
remarked, the Cyberspace Regionalization Project is a unique test of the ability of telecommunications to 
increase the social purposes served by schools. The major goals of the Project are to: 

1. Create an infrastructure of telecommunications to connect two high schools separated by 70 miles 

2. Familiarize and train teachers and students at both schools in its use 
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3. Create programs or activities to bring the students and teachers together 

4. Apply those programs to issues of racial understanding, and 

5. Apply those programs to improvement and reform in the two schools 

These project goals are loosely based on the "Contact Hypothesis" posited as early as 1954 by Gordon 
Allport£. In its most basic form, this hypothesis holds that, under ideal conditions, contact with members of 
different cultural groups promotes positive, tolerant attitudes. These ideal conditions include: 

1 . contact involving persons of equal status 

2. contact taking place under cooperative conditions 

3. contact that is actively supported by powerful authorities 

Cyberspace Regionalization appears to meet all of these conditions, since equal status (students) people of 
different races are working together on a project designed and supervised by teachers and authorized by school 
district administrators. 



IV. MEASURING RACIAL ATTITUDES: The Evaluation Piece 



Four decades of empirical research yielded considerable evidence that contact under the conditions described 
above has beneficial consequences. However, most of that research is now quite dated and often neglected 
considering the racial attitudes of young people and people of African descent. One of the challenges to 
evaluating the Cyberspace Regionalization Project lied in developing updated measures of racial attitudes that 
address salient contemporary issues and that are appropriate for high school-age students. 

A. " Old Fashioned Racism. " Racial attitude research has prompted a number of theoretical orientations and 
alternative measures in the past several decades^. Racial attitude measures were traditionally comprised of 
items attempting to assess what has now come to be known as "Old-Fashioned" or "Dominative Racism". An 
individual with old-fashioned racist attitudes is someone who acts out bigoted beliefs. Prejudice measures that 
tapped social distance, hostility and derogatory beliefs represent that orientation. 

After about 1965, however, standard racial attitude measures had two problems. First, by the middle 1960's, 
most white people knew the socially desirable answers so that the then standard items were more likely to 
trigger politically correct responses than valid attitudes. Second, that generation of items did not correlate well 
with what should have been racially relevant behavior, for example, reported voting intentions or hiring 
decisions. Replacement items were then developed. The new items that correlated best with racially relevant 
behavior were those of an abstract, moral tone, or items that used code words or symbols for blacks. These 
items were thought to tap a new form of racism called "symbolic racism.''^ 

B. " Modern Racism. " Around 1978, led by John McConahay et. al., symbolic racism was re-named as 
"modem racism" to emphasize the contemporary nature. The principal tenets of modem racism are as follows. 

1 . Discrimination is a thing of the past because blacks now have the freedom to compete in the 
marketplace and to enjoy those things they can afford. 

2. Blacks are pushing too hard, too fast and into places where they are not wanted. 

3. The tactics and demands of activists are unfair. 

4. Therefore, recent gains are undeserved. 

5. The prestige granting institutions of society are giving blacks more attention and status than they 
deserve. 

6. Racism is bad. 
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7. TJhe beliefs of modem racism do not qualify as racist because they are alleged to be empirically 
grounded.^ 

Thus, those whose beliefs are described as modem racism do not define their own beliefs and attitudes as 
racist. 



C. " Aversive Racism. " Around 1986, Gaertner and Dovidio developed the concept of "Aversive Racism". 
According to this orientation, many white Americans with strong egalitarian values simultaneously have 
negative feelings and beliefs about blacks. Attitudes need not be consistent and in this case may be the result of 
conflict between cognition and socialization. Because aversive racists put a high value on egalitarian beliefs, 
the contradiction between those feelings and racial attitudes is handled by excluding the racist feelings from 
awareness. Aversive racists also typically avoid close contact with minorities or communicate their underlying 
negative attitudes in subtle, rationalizable ways. Their negativity is likely to be demonstrated in discomfort, 
uneasiness, fear, or avoidance of minorities rather than in outward hostility. The subtlety of this "aversive" 
behavior (in effect, a non-behavior) makes it difficult to document aversive racism through the techniques of 
behavioral research.-Ifi 



D. The Case for Development of a Multi-factor Racial Attitude Assessment Instnunent . Although there has 
been a considerable investment in studying individual and group racial attitudes using the orientations just 
described, the differences among the types have yet to be conclusively demonstrated. That recommends an 
eclectic approach. Additionally, there is a line of research that suggests that racial attitudes are organized 
around content areas or social issues that change over time. Thus, attitudes ebb and flow with variation in 
racial interactions and in social and political events. 

This is the approach adopted for evaluating the Cyberspace Regionalization Project. By piloting and analyzing 
scores on a pool of items from various sources, two multidimensional, multi factor measures of racial attitudes 
were developed.fi The individual items used came from the following sources: 

• The National Opinion Research Center (NORC) at the University of Chicago. All NORC data are based 
on face-to-face interviewing. 

• The Institute for Social Research (ISR) at the University of Michigan. The University's Survey Research 
Center and the Center for Political Studies are noted for their national election analysis. With minor 
exceptions, ISR data are also based on face-to-face interviewing. 

• The Gallup Organization. Gallup employed face-to-face interviewing over most of its history, but 
shifted to telephone interviewing in the late 1980's. 

• CBS/New York Times public opinion polls. 

• Florida State University Professor John Brigham's "Attitude Toward Whites (ATW)" and "Attitude 
Toward Blacks (ATB)" instruments which were developed for and normed on a college student 
population. 

Some of the survey items tap into similar themes or dimensions as previous research about adult attitudes (c.f., 
McConahay and Brigham). For example, a number of questions ask students about interracial relationships. 
Factor analysis allowed us to determine if there was, in fact, an intercorrelation among these items. Factor 
analysis refers to a family of analytic techniques designed to identify components or dimensions, that underlie 
the relations among a set of theoretically linked items. Exploratory factor analysis is used to determine which 
items are meaningfully correlated with the factor presumed to be measured (e.g. interracial relationships). 
Confirmatory factor analysis is applied to estimate the weights of the individual items on the factors. 

Factor analysis of the preliminary data revealed and confirmed the following factors within the two racial 
attitude assessment instruments. Each factor consists of anywhere from three to seven questions. 
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Hunterdon Central CRHS SURVEY 


ASBURY PARK HS SURVEY 


Personal Relations 


White Attitude Expectations 


Social Justice / Anti-Discrimination 


Differences Between the Races 


Old-Fashioned Racism 


Discrimination 


Modem Racism 


Personal Relations I (Social Distance) 


Social Distance 


Personal Relations II (Social Interaction) 



One of the benefits of using a multidimensional assessment tool is that we will not have to rely on individual 
item analysis nor will we have only a single, aggregate racial attitude score. For each student involved in the 
study, we will have a set of racial attitude factor scores with more room for variation and growth estimation 
over time than would be the case with individual item analysis. 

V. BASELINE RESULTS AND RESEARCH AGENDA 

The Cyberspace Regionalization Project Evaluation follows a cohort of students throughout their high school 
experiences and their increasing exposure to Cyberspace Regionalization. The racial attitude instruments were 
administered to all of the ninth-grade students in both high schools in the fall of this past school year 
(1998-99). The ultimate posttest will consist of a re-administration of the instruments to remaining members of 
the cohort shortly before graduating high school. Exposure to Cyberspace Regionalization and intergroup 
contact will be monitored and documented throughout the evaluation period. 

Baseline results revealed a significant amount of variance in student racial attitudes among and between 
groups. The pretest data are mostly a point of comparison against the posttest results, but at least one 
substantive result stood out as particularly interesting. The Hunterdon Central students have less contact with 
people of African descent than the Asbury Park students have with Caucasian people. Said another way, 
Hunterdon Central is more racially isolated than Asbury Park. However, despite being more racially isolated, 
Hunterdon Central students were more comfortable, on average, interacting with students of other races than 
were Asbury Park students. 

VI. CONCLUSIONS 

Cyberspace Regionalization engages two of education's persistent problems — equity and school refomi. The 
activities depart dramatically from the standard menu of imposed programs wrapped in supposed solutions. 

For 40 years, New Jersey has maintained an extensive (and not uncommon) menu of policy initiatives for both 
racial isolation and school improvement — busing, magnet schools, cadres of special teachers and special 
curriculum plus a constant strain for reform in school finance. Each has made some difference but not enough. 
Students remain advantaged or impeded by accidents of birth, economics and geography. 

AT&T and other funding partners are bridging these schools with a wider, more powerful world of 
telecommunications. But how much can telecommunications contribute to central needs for cultural diversity 
and school reform? If students from the two schools become involved in virtual, but still shared activities, will 
there be an effect on attitudes about race? Can the intelligent application of computer-related technology in 
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ichooU address the issue of racial and economic disparity between school districts better than court-ordered 
)usir;g did? 

Answers are likely to lie somewhere between the enthusiasms of technophiles and the cynicism of 
echnophobes. Technology will not make racism disappear. And, teachers do not believe that computer-related 
echnology is the (single) answer to the knotty problem of school reform. Keeping in mind the relative 
dendemess of this (mostly) in-school telecommunications intervention, Cyberspace Regionalization will not 
?e able to change a family's employment circumstances or re-balance the images of commercial television or 
make store clerks polite and accepting; time spent with Cyberspace Regionalization is a small fraction of a 
student's life. How significant that fraction is will be addressed through the evaluation. The important 
questions are not binary ('Yes it does', 'No it doesn't') but rather, how much and how little and under what 
:ircumstances. 

\BOUT THE AUTHOR: Jonathan Becker, J.D., M.Ed. is Projects Director and a research analyst at 
[nteractive, Inc., a technology development and evaluation firm in Huntington, NY, and a doctoral student in 
:he Politics and Education Program at Teachers College, Columbia University. 
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Introduction 

At the Secretary's conference on evaluating the effectiveness of educational technology we will be asked 
to address the following fundamental questions: 

How does technology impact student learning? 

What can we know about the relationship using data and tools available? 

What can we learn about the relationship in the future with new tools and new strategies? 

The conference will highlight new and emerging data on effectiveness of technology in primary and 
secondary education reflected in the latest research and promising practices. The intent of the 
proceedings is to influence the way educators, teachers, policy makers evaluate and assess the growing 
investment in technology and to provide schools with tools and strategies for effective evaluation. 

In this paper we hope to inform the discussion by discussing recent changes in evaluation theory and 
practices, and by clarifying some definitions of evaluation, technology and student learning. It is evident 
that there are multiple definitions of evaluation, of technology and of student learning and theses multiple 
definitions must be engaged prior to substantive debate over the course of future directions. We will 
highlight what we believe are instances of promising practices and conclude with a list of 
recommendations concerning the evaluation of the effectiveness of technology in teaching and learning. 

Recent Changes in Evaluation Practices 

We should say at the outset that evaluation means many things to many people. According to Glass and 
Ellett (1980) "evaluation- more than any science- is what people say it is, and people currently are saying 
it is many different things" (cited in Shadish, Cook and Leviton, 1991, p. 30). In a recent examination of 
evaluation practice, we are encouraged to bring a critical eye to bear on the purpose and conduct of 
evaluations. Shadish Cook and Leviton (1991) recommend that in any evaluation endeavor we ask 
fundamental questions about five key issues: 17^ 

1 BEST COPY AVAILABLE 
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1. Social programming: What are the important problems this program could address? Can the 
program be improved? Is it worth doing so? If not, what is worth doing? 

To maximize helpful change in the public interest, is it more effective to modify the philosophy or 
composition of whole programs, or to improve existing programs incrementally-perhaps by 
modifying regulations and practices, or influencing which local projects are phased out? Should 
the evaluator identify and work with change agents, or merely produce and explain evaluation 
results without forming alliances with change agents? Should evaluators try to change present 
programs or test ideas for future programs? Under what circumstances should the evaluator refuse 
to evaluate because the relevant problem is not very important or the problem is not likely to 
ameliorate the problem? 

2. Knowledge use: How can I make sure my results get used quickly to help this program? Do I want 
to do so? If not, can my evaluation be useful in other ways? 

Should conceptual or instrumental use have priority? Should the evaluator identify and attend to 
intended users of evaluations? If so, which users? What increases the likelihood of use, especially 
for instrumental versus conceptual use? 

3. Valuing: is this a good program? By which notion of "good"? What justifies the conclusion? 

By whose criteria-of merit should we judge a social program? Should prescriptive ethical theories 
play a significant role in selecting criteria of merit? Should programs be compared to each other or 
to absolute standards of performance? Should results be synthesized into a single value judgment? 

4. Knowledge construction: How do I know all this? What counts as a confident answer? What 
causes that confidence? 

How complex and knowable is the world, especially the social world? What are the consequences 
of oversimplifying complexity? Does any epistemological or ontological paradigm deserve 
widespread support? What priority should be given to different kinds of knowledge, and why? 
What methods should evaluators use, and what are the key parameters that influence that choice? 

5. Evaluation practice: Given limited skills, time, and resources, and given the seemingly unlimited 
possibilities, how can I narrow my options to do a feasible evaluation? What is my role-educator, 
methodological expert, judge of the program- worth? What questions should I ask, and what 
methods should I use? 

What should the role of the evaluator be? Whose values should be represented in the evaluation? 
Which questions should the evaluator ask? Given limited time and resources, which methods 
should be used to best answer the questions? What should the evaluator do to facilitate use? What 
are the important contingencies in evaluation practice that guide these choices? 

Experts on program evaluation (House, 1993; Schorr, 1997; Shadish, Cook and Leviton, 1991) all 
indicate that program evaluation has undergone a major transformation in the last three decades. It has 
changed from "monolithic to pluralist conceptions, to multiple methods, multiple measures, multiple 
criteria, multiple perspectives, multiple audiences, and even multiple interests. Methodologically, 
evaluation moved from primary emphasis on quantitative methods, in which the standardized 
achievement test employed in a randomized experimental control group design was mostly highly 
regarded, to a more permissive atmosphere in which qualitative research methods were acceptable 
(House 1993. p. 3) The most fundamental shift has been away from a blind faith in the science of 
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evaluation and experimental research methods based on standardized test scores. These changes in the 
practice of evaluation have significant implications for questions about the future of the evaluation of 
technology and student learning outcomes. 

The primary question to which we always turn to is: How does technology impact student learning? We 
don't, however, make implementation decisions based on this question. What do we know about this 
relationship using data and evaluation tools currently available and what could we learn in the future 
about technology and student learning assuming the application of new evaluation tools and strategies? 
The answer to the first question is fairly straight forward: The relationship depends on how you define 
student learning and how you define technology. 

If one defines student learning as the retention of basic skills and content information as reflected on 
norm referenced and criterion referenced standardized tests, then, evidence suggests, there is a positive 
relationship between certain types of technology and test results. For instance, it is well established that if 
a teacher uses computer assisted instruction or computer based learning approaches, where the computer 
is used to manage the "drill and skill'' approach to teaching and learning, students will show gains on 
standardized test scores. This view of technology reduces the equation to only a s.udent, a computer and 
a test. It ignores the effects of schools, teachers, and family and community life on the learning process. 
Even tough we cannot control for these variables, we must not discount them. 

If, on the other hand, one views the goal of education as the production of students who can engage in 
critical, higher order, problem-based inquiry, new potential for entirely different uses of technology 
emerge. For instance, the world wide web can be used as a source of information from which students 
can draw to solve real world problems by applying technology knowledge and skills. We can evaluate 
these outcomes but it is more complicated than the standardized testing route. Standardized tests are an 
efficient means for measuring certain types of learning outcomes but we must again ask ourselves, are 
these the outcomes we value for the new millennium? To a certain extent we are living out the decisions 
reflected in previous evaluation methods which constrain our thinking about the purpose and 
effectiveness of technology in education. 

Policymakers, evaluators and practitioners may have vary different answers to fundamental questions 
about the effectiveness of educational technology. Everyone is asking for results of the investment of 
technology in education. Perhaps the primary difficulty in coming up with new ways of evaluating or 
assessing the impact of education technology is that there is little consensus about its purpose (Trotter, 
1998). Policy makers often work from a cost-benefit model with increases in norm referenced and 
criterion referenced test scores viewed as the primary benefits. This appears to be at odds with the view 
held by teachers or by the public that educational technology benefits include preparing students for jobs, 
increasing student interest in learning, increasing student access to information and making learning an 
active experience (all rated above technology's impact on basic skills by parents in a 1998 public opinion 
survey sponsored by the Milken Exchange). 

The question really should not be does educational technology work? "but when does it work and under 
what conditions?" (Hasselbring cited in Viadera, 1997). In practice, student achievement outcomes are 
mediated by the processes of teacher integration of technology into instruction. Technology can be used 
to improve basic skills through automated practice of drill and skill. Technology can also be used to 
facilitate changes in teacher practices that promote critical, analytic, higher order thinking skills and 
real-world problem solving abilities by students. The ability of teachers to foster such changes depends 
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significantly on training that shows them how to integrate technology into content specific instructional 
methods. This has been shown through programs such as the Adventures of Jasper Woodbury conducted 
at Vanderbuilt University, the national Geographic Society's Kid's network, and work done at University 
of Massachusetts, MIT and TERC with Simcalc. 

Any innovation in our system of education, including technology, raises persistent questions about the 
purposes of education. Is it to provide training in fundamental and basic skills? Is it to prepare students 
for the work force? Is it to produce citizens for an effective democracy? Is it to produce an equitable 
society? Is it to produce broad, life-long learners? Is it to prepare students with critical thinking skills for 
a complex new world? According to educational researcher Larry Cuban, unless educational policy 
makers can agree and clarify the goals for using technology, it makes little sense to try and evaluate it. 

This raises questions about assessment and evaluation of educational technology. Do traditional, 
standardized assessments measure the benefits that students receive from educational technology? In the 
evaluation of social programs in general, the profession of evaluation has moved away from standardized 
test scores as a meaningful measure of the impact of programs. Evaluation theorists like Mackie and 
Cronbach have argued that there are too many critical relationships occurring in social phenomenon to be 
adequately captured by the traditional experimental design. "Social programs are far more complex 
composites, themselves produced by many factors that interact with one another to produce quite 
variable outcomes. Determining contingent relations between the program and its outcomes is not as 
simple as the regulatory theory posits" (House, 1993, p. 135-6). Besides improvements in retention of 
x rote facts, technology can improve student attitudes toward the learning process, perhaps we should be 
assessing actual, authentic tasks produced through the processes of student interaction and collaboration. 
Perhaps we should be developing technologically based performance assessments to measure the impact 
of technology on student learning. 

We have been fairly successful in determining the impact of technology on basic information retention 
and procedural knowledge. However, we have been less than successful in evaluating the impact of 
educational technology on higher order or metacognitive tinning skills. 

Needed: New and Expanded Definitions of Student Learning Outcomes 

What are needed more than anything else are a new set of clear learning outcomes for students who must 
live in a complex world. New learning outcomes must focus on the demands of the new world 
environment. We need students who can think critically, solve real world problems using technology, 
take charge of their life-long learning process, work collaboratively and participate as citizens in a 
democracy. Experts in the area of technology and education such as Jan Hawkins and Henry Becker have 
provide ideas that could be developed into criteria for new ways of thinking about technology, teaching 
and learning. These new learning outcomes could be translated into learning benchmarks and new types 
of assessment and methods for measuring outcomes could developed to measure these benchmarks. 

What we are looking for is a transition from isolated skills practice to integrating technologies as tools 
throughout the disciplines. Jan Hawkins argued that to realize high standards, education needs to move 
beyond traditional strategies of whole group instruction and passive absorption of facts by students. New 
more effective methods are based on engaging student in complex and meaningful problem-solving 
tasks. Technologies need to be used to bring vast information resources into the classrooms. We need a 
transition from inadequate support and training of teachers to support for all teachers to learn how to use 
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technologies effectively in everyday teaching (Hawkins, 1996). 

According to Becker (1992) in an ideal setting, teachers use a variety of computer software, often 
working collaboratively to address curricular goals. Students exploit intellectual tools for writing, 
analyzing data, and solving problems and they become more comfortable and confident about using 
computers (Becker, p. 6). Exemplary teachers use computers in lab settings as well as classroom settings 
at the school for consequential activities that is where computers are used to accomplish authentic tasks 
rather tan busywork such as worksheets, homework assignments, quizzes or tests. Means and Olson 
(1994) outline a set of criteria for successful technology integration projects: An authentic challenging 
task, a project where all students practice advanced skills, where work takes place in a heterogeneous, 
collaborative groups, the teacher acts as coach and provides guidance, and where work occurs over 
extended blocks of time.. 

Evaluating for New Visions of Technology Teaching and Learning 

It is clear that teaching and learning processes are embedded within complex systems. The challenge is to 
develop evaluation models that reflect this complexity. Just as technology has caused us to reevaluate the 
nature of knowledge and instruction, it prompts us to reevaluate the forms of evaluation that are brought 
to bear when examining educational technology. According to Schorr (1997) we need a new approach to 
the evaluation of complex social programs, one that is theory-based, aiming to investigate the project 
participant's theory of the program; one that emphasizes shared rather than adversarial interests between 
evaluators and program participants; one that employs multiple methods designs; and, one that aims to 
produce knowledge that is both rigorous and relevant to decision-makers. In order to accomplish these 
tasks it will be necessary to design evaluations of technology in K.-12 settings based on the experiences 
of evaluators, the experiences of program developers, "state of the art" in the field of technology and 
learning and the various program descriptions. 

Several studies and reports have done an exemplary job at pointing us in promising directions for future 
evaluations of the effectiveness of educational technology. For instance Bodily and Mitchell have 
prepared an evaluation sourcebook for "Evaluating Challenge Grants for Technology in Education" 
published by the RAND Corporation. Bodilly and Mitchell (1997) acknowledge that the outcomes sought 
in technology infusion projects are complex and "not entirely captured by traditional educational 
measures, seeking better learning outcomes "on a complex variety of dimensions rather than 
improvements in traditional test scores" but they go on to recommend that some stake holders may be 
interested in test scores as measures of student learning. They indicate that performance outcomes are the 
results of complex causes. Technology may be only one of many input variables causing changes. A 
project's implementation and outcomes are heavily influenced by its context. Goals of various 
educational technology projects are unique and may not be captured by a uniform evaluation design and 
multiple evaluation design are required. 

In terms of outcome goal, they include a wide variety of possibilities beyond traditional test scores 
including: short term changes in student outcomes like disciplinary referrals, homework assignments 
completed or longer term indicators such as changes in test scores or student performances, increased 
college going rates, increases in job offers to students. Other outcomes are defined as higher order 
thinking skills, more sophisticated communication skills, research skills, and social skills. More 
sophisticated outcome measures must be located or developed by evaluators in order to gauge new 
effects of technology on learning. 
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Other outcome measures might be found in participants' (teachers and students) perceptions about the 
implementation, quality and benefits of the program. These might reflect student engagement levels as 
well as satisfaction levels. Other interim performance indicators might include the effect of the proOgram 
on community and family participation or involvement, and student and teacher retention. Declines in 
disciplinary referrals and special education placements may also serve as outcome measures. The federal 
government, state departments of education, school district or schools might develop criteria for 
standards of good practice indicators and associate learning outcome benchmarks. 

Other indicators of student outcomes such as higher order thinking skills and ability to apply knowledge 
in meaningful ways might be measured by performance assessments, portfolios, learning records, and 
exhibitions. Of course norm referenced and criterion referenced assessments can also supplement these 
alternatives outcomes. School districts are encouraged to use multiple and varied measures of outcomes. 
Student performance indicators such as attendance, reductions in drop-out rates, successful transitions to 
work and post-secondary institutions should be considered. Baseline data should be established at the 
beginning of the project. They also propose that a list of common indicators across projects be used as a 
tool for summative program evaluation. 

Bodilly and Mitchell refer to work on the evaluation of technology in educational reform conducted by 
Herman (1995) and Means (1995). They conclude that broad-based technological reforms, those that 
attempt multiple changes in a school besides the insertion of a single computer-based course, such as an 
attempt to create a constructivist curriculum across all grade levels supported by computer technology 
are more difficult to measure in terms of outcomes. They state: efforts to trace the effects of these 
projects must take into account measuring effects in dynamic situations where many variables cannot be 
controlled and where interventions and outcomes have not been well defined for measurement" (p. 16). 
They also assert: "The complex environments in which technology projects are embedded make 
inference of causal relations between project activities and outcomes tenuous" ( p. 20). 

Implementation analysis becomes important under these conditions. With all of these complexities, 
effects of technology on student outcomes may not occur in the short-term evaluations must take into 
account the different phases of a schools integration of technology: purchasing and installing hardware 
and software, training teachers, integrating technology into the curriculum and instruction. Evaluation 
designs must therefore, be longitudinal in design and account for changes in the target population. 
Tracking comparison groups not exposed to technology or using national surveys to assess the likely 
level of background effects will often be necessary. 

CMC corporation conducted a two year evaluation of the Boulder Valley Internet Project. The project 
employed a variety of evaluation method and developed a theoretical tool. The Integrated Technology 
Adoption Diffusion Model, to guide the evaluation. Evaluations should include the contexts within which 
technological innovations occur. This includes looking at technological factors, individual factors, 
organizational factors and teaching and learning issues (See Sherry, Lawyer-Brook, and Black, 1997). 
Evaluation designs must be flexible enough to attend to the varying degrees of adaptation occurring with 
different content areas. Evaluations must include implementation assessments, formative assessments as 
well as standard summative and outcomes assessments. Evaluations must include the quality of training 
programs offering teachers the opportunity to learn new technologies within relevant, subject-specific 
contexts. 



Recommendations 
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We need to take a more formative approach to the evaluation of technology because of the rate of change 
in technologies. Technology changes so quickly that teachers are often asked to keep up and integrate 
new ideas at break neck speeds. The definition of what is the innovation is thus constantly at issue and 
we must spend time documenting the program which may be changing over time.. 

In order to get at the complexities of these processes multiple measures (quantitative and qualitative) 
should be used. These should include traditional experimental and quasi-experimental designs and 
include such methods as paper surveys, email/web-based surveys, informal and in-depth interviews, 
focus group interviews, classroom observations and document analysis. 

Evaluation design should incorporate longitudinal studies of cohorts of students over several years. In 
addition evaluation designs should rely less of participants self reported attitudes and more on 
observations of participants actions within learning contexts. We need to be in classrooms to observe 
how teachers are incorporating technology into their instruction and what effect this is having on student 
learning processes. We would recommend further efforts such as those by Milken and Elliot Soloway, to 
improve the format for research designs to allow for comparisons across sites. 

Future evaluations should not focus on simple outcomes measures such as posttests but should also focus 
on complex metrics describing the learning process such as cognitive modeling (Merrill, 1995). Research 
and evaluation needs to demonstrate the potential of educational technology but in a way that attends to 
the layers of complexity that surround the processes. We need to include a wide variety of experts and 
stakeholders. 

Conduct implementation evaluations prior to outcomes evaluations. Spend time necessary to determine 
whether an innovation as been adopted or fully implemented before trying to determine its effectiveness. 



Focus on description of the program, treatment, or technological innovation, develop stronger 
descriptions of how the technological innovation is configured. 



Recognize the complexity of educational technology; Define technology as an innovative process linking 
teaching and learning outcomes rather than a product which is dropped into the black box of teaching and 
learning outcomes defined as improvements on standardized test scores. Reduce the reliance on 
standardized test scores as the primary evaluation outcome. Replace dogmatic applications of 
experimental designs with designs that allow us to view the complexity of technology based reforms of 
teaching and learning from multiple perspectives. Adopt multifaceted approaches to evaluation that 
include case studies and theoretical modeling which includes individual, organizational, technological 
and teaching/learning aspects of adoption and diffusion of innovations. This means that participant 
observation of programs will be used as a form of data collection. This type of data collection is not 
inexpensive but provides evidence beyond self reported data or gross outcome measures like test scores. 



// //// 



[The Cyberspace Regionalization Project:! 
and Home Learning Technologies! 




[Measurement Issues with Instructional 
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Measurement Issues with Instructional 
and Home Learning Technologies 

I. INTRODUCTION 

II. WHAT ARE WE MEASURING? 

III. MEASURING USE OR EXPOSURE 

IV. HOW DO WE KNOW IF TECHNOLOGY WORKS? MEASURING THE DEPENDENT 
VARIABLE(S) 

V. CONCLUSION 
APPENDIX 

Ten Practical FAQ's (Frequently Asked Questions) about measuring IT effects 
Selected Sources on Measurement of Instructional Technology 

I. INTRODUCTION 

Evaluating the effects of technology use provokes the same evaluation challenges as does any other 
program intervention. The issues that I address in this paper are based upon my experience in evaluating 
the achievement effects of specific technology implementations. The five studies that have offered me 
the largest learning laboratory are listed in Table 1 . Each required a careful description of the technology 
to be studied, a measure of how much students used the technology, and a measure of achievement gains. 

As Mann has pointed out in "Documenting the Effects of Instructional Technology: A Fly-Over of Policy 
Questions", a variety of stakeholders are beginning to ask questions about technology use in schools. 
Many of these questions go no further than "Does technology work?" Or, "Does technology use improve 
student achievement?"; "Is technology in schools worth the money it costs?"; "Are there benefits to 
students beyond achievement?" 



Table 1 : 

Studies of Technology Use and Student Achievement 



Study 


Purpose 


Sample/ 


Method and Data 


Findings 






Setting 


Collection 
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The Cyberspace 
Regionalization 
Project: “Virtual 
Desegregation” 


Can audio-visual 
telecommunications be 
used to bridge gaps of 
geography, race and 
social class? 


650 9th grade 
students in two 
high schools: one 
upper income 
with Caucasian 
students; one 
lower income with 
students of 
African descent. 


Four year study. 
Interviews, 
surveys, annual 
pre-post 

administration of 
a Racial Attitude 
Assessment 
Instrument, 
administrative 
data transfer. 
Four year data 
collection. 


Lightspan Achieve 
Now and the 
Home-School 
Connection: 
Adams 50, 
Westminster, CO. 


Does implementation of 
a game-like, 
CD-ROM-based, K-6 
curriculum launched at 
school and used at 
home with families 
improve student 
achievement in math 
and language arts? 


elementary 
schools; 2,000+ 
students and 55 
teachers in grades 
2-5. 


Three year study 
of 3 elementary' 
schools using 
Lightspan 
compared with 3 
not using 
Lightspan. 
Annual pre-post 
Terra Nova data, 
district reading 
test scores, 
Colorado test 
scores. 

Observations in 
classrooms. 
Interviews with 
parents, teachers, 
and students 
four times each 
year. Learning 
Combination 
Inventory. 
On-line data 
collection. 
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Study in progress. 
However, baseline 
data collected in 
the Fall of 1998 
reveal gaps in 
interracial contact 
and significant 
variation in racial 
attitude scales. 



After one-year of 
implementation, 
the students in 3 
treatment schools 
surpassed students 
in the control 
.schools and 
significantly 
outperformed them 
on the CTB-Terra 
Nova (Reading and 
Math). 
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Read 180 
Scholastic, Inc. 
(National, urban 
settings) 


Can a CD Rom 
interactive basic skills 
curriculum remediate 
prior deficiencies for 
early adolescents who 
are 4 or more grades 
behind in achievement? 


Random 
assignment of 
1,400 6th and 7th 
grade students to 
Read 180 and 
control classrooms 
in 7 big city 
school districts 
(Chicago, Dallas, 
Miami- Dade, 
Houston, Atlanta, 
San Francisco, and, 
Boston) 


Two year 
pre-post 
measures in 
Stanford 9 
Language Aits 
subtests in Read 
1 80 and control 
classrooms. 

Self efficacy, 
discipline, 
achievement in 
other subject 
areas, and 
attitude toward 
school ace 
examined. 


Data collection 
begins September \ 
1999. 


Technology Impact 
Study in the 
Schools of the 
Mohawk Region, 
New York State 


What is the impact on 
student achievement 
associated with a $ 1 4 . 1 
million investment in 
educational technology? 


55 school districts, 
4,041 teachers, 
1,722 students, 

159 principals, 41 
superintendents 


Teacher survey, 
principal survey, , 
administrative 
data transfer of ; 
New York State 
PEP and Regents 
test scores 


For the schools that, 
had the most 
technology and 
training for 
teachers, the 
average increase in 
the percentage of 
students who to 
ok and passed the 
Math Regents 
Exam was 7.5; the 
average increase 
for the English 
Regents Exam was 
8.8. 


West Virginia’s 
Basic 

Skills/Computer 
Education (BS/CE) 
Program 


What effect does a $70 
million statewide 
comprehensive 
instructional technology 
program have on 
student achievement? 


1 8 elementary 
schools, 950 fifth ■ 
grade students, 
teachers and 
principals in all 
the schools 


Teachers survey 
Principal survey 
Student survey 
Observations 
Interviews with 
principals, 
teachers, and 
students 
Stanford 9 data 
for two years 


A BS/CE 
technology 
regression model 
accounts for 1 1% 
of the total 
variance and 33% 
of the within 
school variance in 
the one-year basic 
skills achievement 
gain scores. 
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II. WHAT ARE WE MEASURING? 

Many of the program administrators responsible for IT have not thought through the questions they want 
answered by documentation research, nor can they be expected to since operational responsibilities often 
preempt evaluation. Part of the job of the evaluator is crafting work that serves the needs of the 
stakeholders: Is this an evaluation for re-funding? For use in curriculum refinement? For analysis of 
classroom instruction? For public relations? For all of these? 

Because stakeholder needs are not always clear, the first measurement challenge is to determine the 
technology "input" to be examined. Technology is lots of things: computers, CD-ROM and videodisc 
players, networked applications. If we focus on computers, it generally is not the use of the computer per 
se that is of interest, but rather a specific use, especially particular software. 

For most readers of this paper, the "what is the technology" question will seem elementary. However, my 
experience has been that many stakeholders — particularly school administrators, school board members, 
and legislators — expect that if hardware is purchased, then improved achievement should follow. A 
common situation we have faced is being asked to determine achievement gains in schools where 
computers and word processing software are purchased. The notion that doing anything on a computer 
should lead to (any) achievement gains is widespread. (We were once asked to measure the math 
achievement impact of having provided Corel's WordPerfect word-processing software to all the 
elementary teachers of a district!) Therefore, identifying what technology use is being analyzed is a first 
step, and a step I would not bother to relate had 1 not learned the hard way that identifying the technology 
to be measured requires a considerable amount of interaction with stakeholders. 

Is the technology question really a focus on the teaching efficacy of a particular software that students 
are using? If so, is there a relationship between the software design characteristics and student 
achievement? Do any of the following make a difference: instructional control, feedback, objectives and 
advance organizers, cognitive strategies, conceptual change strategies, scaffolding of learning support, 
still and animated graphics, dynamic visualization, video, navigational technique, text and story content, 
game context and visual metaphor fantasy context, Window presentation styles? 

Or, is the question about multiple sites for technology use? The home? The school? Both? And if so, how 
much of what interaction in which site is related to achievement? 

Do different technologies result in different kinds of achievement? For instance, do telecommunication 
distance learning technologies such as access to online resources, document exchange and discussion, or 
professional development on-line improve student achievement? If they do, is this be a direct 
relationship? How would we isolate these uses while examining student achievement? 

It is easy to see how an initially simple question like, "What is the relationship between technology use 
and student achievement?" blossoms into refinements and further definitions. Carefully defining the 
technology to be studied then takes us to the next step. 



III. MEASURING USE OR EXPOSURE 



Just because technology is present does not mean that the students are using it. How do we measure the 
intensivity of student use? 



We faced this question in every study we have done. We have used observations, file server records, 
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student reports, parent reports (thousands of telephone interviews, each logged and coded), teacher 
reports, and on-site observations. Because it isn't feasible to shadow every student every day, 
observational data, although probably both reliable and valid, is not often feasible. Metering and file 
server records, although able to record time on the computer or software, are not available in most 
schools. The next level of data is self report data from students, which can be verified by teachers and 
parents. If we are examining the relationships between the use of some technology and student 
achievement, we do sampled surveys of use. We ask students, teachers, and parents about the previous 
day or week's activity. We use e-mail, web-site, telephone, face-to-face, and paper and pencr- surveys to 
document student use. 

Not surprisingly, filling out surveys is not a priority for many educators, whether they are sent by e-mail, 
snail mail, or over telephone lines, but we have always had excellent cooperation that easily exceeds the 
minimum standards for sample size and response. Student reports of their own behavior tend to be more 
accurate than parent or teacher responses, although children younger than fifth grade often have 
difficulty estimating time. Teachers are usually able to tell us how much in-class time that students spend 
on the computer, although it often depends on which day, which class, and which student. Teacher 
reports are aggregate reports, while student reports are specific to the individual student. 

Because student use (at least in schools) is related to teacher use of and comfort with technology, we 
include in the description of the technology the amount of teacher professional development and 
integration into the curriculum. We ask teachers and administrators about use. We examine teacher 
professional development participation, both in school and out of school, formal and informal. Self 
reports of technology literacy, faculty meeting agendas, lesson plans, and observations all help to 
describe what the teacher knows about technology, how comfortable the teacher is with technology, and 
how and how often the teacher is able to integrate technology into the curriculum. 

IV. HOW DO WE KNOW IF TECHNOLOGY WORKS? MEASURING THE 
DEPENDENT VARIABLE(S) 

While this paper is about measurement issues and student achievement, there are worthy reasons to use 
technology beyond bottom-line achievement. We have examined technology use and self efficacy, 
attitude about school, attendance, and discipline. 

However, to understand the relationship between technology use and student achievement, we are most 
comfortable with examining gains in individual student achievement that would be reasonably expected 
because of the technology. Thus, we don't expect that time using music composition software would 
accelerate student learning in biology. The measures used must relate to the expectations of the 
technology. \ 

We use the same data that schools use to determine achievement, even when we might not think it is the 
best form of measurement. We use these data because that is how the districts and their superordinate 
jurisdictions measure achievement. While we can argue that most achievement tests do not accurately or 
fully explain what students leam, the reality is that achievement data is often the best we have. 

Thus we often rely upon gain scores from September to May on norm referenced tests such as the 
Stanford 9, the Iowa Tests of Basic Skills, or CTB-Terra Nova. Since most districts don't test twice a 
year, this usually requires some negotiation. However, the result is that we have individual student gain 
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scores to relate to the individual student use measures. 

Additionally, we use grade, teacher developed tests, state achievement tests, district achievement tests, 
and authentic displays of student work. The more types of data, the better the understanding. 

V. CONCLUSION 

If you look across the measurement literature (and Jay Sivin-Kachalan and Ellen Bialo have, see sources 
below), you will find different methods to study different combinations of different interventions. It is 
hard to make those disparate studies add up in a way that compels belief. In part, that is the nature of 
decentralized science in a democracy. Still, we would like to see a short list of preferred evaluation 
methods or models, each for example, with two alternative methods for different intervention niches like 
early childhood literacy or gender studies of literacy applications delivered on the Internet. We would 
like to see those models developed and recommended (or even encouraged) by funding agencies. That 
way, at least some of what we do would add up in a more direct fashion than has so far been the case. 

Measuring technology outcomes is undeniably messy and imperfect. It is also important for the 
practice-improving signals "hat can be developed even from this sometimes frustrating enterprise. It may 
also be helpful to recognize that just as instructional technology continues to evolve and to improve, so 
does our ability to document inputs and measure effects. 

About the Author: Charol Shakeshaft. Ph.D., is professor in the Department of Administration and 
Policy Studies, School of Education, Hofstra University, Hempstead NY 11590. An internationally 
recognized expert in gender studies and women's leadership in school administration, Professor 
Shakeshaft's new book is In Loco Parentis: Sexual Abuse in the Schools (San Francisco, Josey-Bass, in . 
press). Dr. Shakeshaft is a Managing Director of Interactive, Inc., 326 New York Avenue, Huntington, 
NY 11743-3360: p 516 5470464: f 516 547 0465. 



APPENDIX 

Ten Practical FAQ's (Frequently Asked Questions) about measuring IT effects 

1 . Q: It is too early to expect results. A: It is always too early but if there is a partial implementation 
(which is almost always the case anyway) then we need sensitive measures and an expectation of 
probably faint signals of effect. 

2. Q: Instructional Technology wasn't the only thing we did. We changed textbooks, moved to a 
house plan, etc. A: Good, there are no single answers, not even technology. If the documentation 
plan calls for measuring the different dimensions of all the things that were going on, then 
regression analysis will allow testing for differences in the strength of relationships between 
different input clusters and outcome measures. 

3. Q: We changed tests two years ago. Can we still look for effects? A: Everybody changes tests and 
that is more of an inconvenience to the analyst than a barrier to inquiry. The whole point of 
nationally normed tests is to facilitate comparison. 

4. Q: We keep changing and replacing both hardware and software. How can we know which version 
of what makes a difference? A: That's an excellent question. We all need to do a better job of 
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keeping track of what hardware/software experiences which kids had. 

5. Q: Doesn't it take thousands of cases to do good research? Our district(school) isn't that big? A: 
With well constructed samples, it is possible to generalize to the population from surprisingly 
small numbers of respondents. Selecting those sampling dimensions (and getting access to schools, 
teachers and children) is one of the places where the client organizations can be helpful. 

6. Q: How can you say for sure that IT "caused test score gains"? A: Strictly speaking, none of us can 
make that claim on the research designs that are practically feasible. But social science research is 
seldom if ever causal. One way or the other, decision makers have to commit their organizations. 
We try to help with the best data from the most powerful designs we can get. 

7. Q: If somebody outside the school district pays for the study, then it isn't objective. A: We do lots 
of studies paid for by third parties. The question is not, who paid for it, but how was it done. We 
always report our methods (sample, data collection instruments and techniques, analysis 
procedures) and we make that publicly available. If everyone follows the rules of science and if the 
study followed those rules, then the objectivity is there regardless of the auspices. 

8. Q: It takes millions of dollars to do good research. A: Research that ends up with compelling 
results is sometimes costly. But we find that districts and schools will help with data collection, 
they do part of the work of mailing, they critique procedures and generally share costs to make 
things feasible at modest prices. 

9. Q: The most important question is, does IT change the act of teaching? How can you find that out? 
A: We believe in multiple methods. That's why most of our work is quantitative/qualitative (or 
vice versa) in successive waves. Lots of people think that IT can help teachers use more 
constructivist methods and we have been developing and refining item banks to measure just 

that — the shift from instructivist to constructivist. 

10. Q: Evaluations are always ignored. A: Some are. It depends on how directly (and simply) the 
reports and the underlying data speak to the policy issues. And also on the patience of the policy 
makers and of the measurement people. 

Selected Sources on Measurement of Instructional Technology 

• International Society for Technology in Education (ISTE) (1998). National Educational 
Technology Standards for Students. Eugene, Or. (frinded by the National Aeronautics and Space 
Administration (NASA) in consultation with the U.S. Dept, of Education; the Milken Exchange on 
Education Technology; and Apple Computer, Inc.) (www.iste.org). 

• The CEO Forum on Education and Technology (1997). School Technology and Readiness Report: 
From Pillars to Progress. Washington, D.C. (www.ceofomm.org). 

• Milken Exchange on Education Technology. (1998). Seven Dimensions for Gauging Progress. 
Santa Monica, CA. (www.mff.org). 

• Sivin-Kachala, J. & Bialo, E.R. (1999) (For the Software & Information Industry Association). 
1999 Research Report on the Effectiveness of Technology in Schools. Washington, D.C. 
(www.siia.net). 
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In 1994 the Idaho Legislature passed HB 901, the Idaho Technology Initiative. The 
initiative provided for one-time and ongoing monies for the purchase and integration of 
technology into the K-12 public school systems. Between the years 1994 and 1998, $10.4 
million per year has been allocated to Idaho’s K-12 public school systems, an average of 
$42.55 per student, per year, or $212.75 per student over the five-year period. 

In addition to providing monies for technology, the initiative provided for the creation of 
an administrative council to be established under the State Board of Education. The 
administrative council, named the Idaho Council for Technology in Learning (ICTL), created 
eight specific goals for technology in Idaho’s schools by which to evaluate the impact of the 
State’s investment, and fulfill the Legislative charge which specifically requires the ICTL to 
accomplish the following: 

1) Develop statewide performance indicators 

2) Address impact, costs, and benefits of projects funded through the ICTL 

3) T rack progress and funds spent to achieve indicators 

This report explains the design of the plan created to accomplish this charge which 
uses statewide tests, research studies, student surveys and technology examples to measure 
the effectiveness of technology in the Idaho’s public schools. The following questions were 
asked and answered: 

Goal - Integration of Technology 

Have students improved their academic performance as a result of the integration of 
technology in Idaho’s K-12 schools? 

Answer - There is a positive relationship between academic performance in core 
studies, language, math, and reading and the integration of technology in Idaho's K- 
12 schools (See Page 3). 

Have Students improved their technological literacy as a result of exposure to technology? 

Answer - Both 8 th and 11 th Grade groups increased their technological literacy as a 
result of exposure to technology (See Page 20). 

What technology-related factors have the greatest impact on academic gain in Idaho’s 
schools? 

Answer - The combined top six factors for 8 th and 11 th grade students are: the ability of 
the student to choose the appropriate software tool for completing a project, amount of 
computer use at school, exposure to Internet and email use, amount of computer usi 
at home, use of technology for class projects, and use of software to simulate “real-life” 
experiences (See Page 22). 
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Goal - Compatibility 

Have schools been able to share information and resources through the use of 
technology? 

Answer - Results from the school districts revealed extensive use of Internet and 
email by both teachers and students in addition to moderate use of Idaho’s Distance 
Learning facilities (See Page 23). 

Goal - Teacher Preparation 

Have schools and colleges of education worked together to effectively prepare teachers to 
teach using technology? 

Answer - Schools and colleges of education have worked together to effectively 
prepare teachers to teach using technology (See Page 26). 

Goal - Collaboration with Communities and Businesses 

Have schools involved community members, businesses, and postsecondary institutions in the 
implementation and use of technology in schools? 

Answer - Community members, businesses, and postsecondary institutions have made 
significant investments and have been involved in the implementation and use of 
technology in schools (See Page 30). 

Goal - Technology Systems Enhancing the Efficient Operation of Idaho Schools 

Have schools used technology to improve administrative efficiency in school operations? 

Answer - Schools have used technology such as email, Internet, and electronic data 
storage to improve administrative efficiency in school operations (See Page 32). 

Goal - Training of Students to Maintain Technology 

Are students able to install, maintain, and support technology? 

Answer - Students have been given the opportunity through the Technology Support 
Technician Program to install, maintain, and support technology (See Page 34). 

This report clarifies issues surrounding the debate over whether the use of 
technology in Idaho’s schools has had a positive impact on students. The benefits of 
technology in teaching and learning are clear: increased academic achievement, 
improved technology literacy, increased communication, well-trained, innovative 
teaching, positive relationships with the community, more efficient operation of schools, 
and technically qualified students ready to enter today’s workforce. 

By appropriating funds, through the Idaho Technology Initiative, the legislature has 
made a valuable investment in Idaho’s future which has paid and will continue to pay great 
dividends as shown in comparisons between Idaho and the nation. Moreover, the funds 
distributed to districts thus far have made it possible for Idaho schools to create a base upon 
which to build infrastructures such as technology hardware and software, compatible network 
infrastructure, timely teacher and student training which are key to providing for our students’ 
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needs in the 21 sl century. 

Introduction 

In 1984 the Idaho Legislature passed HB 901, the Idaho Technology Initiative. The 
initiative provided for one-time and ongoing monies for the purchase and integration of 
technology into the K-12 public school systems. Between the years 1994 and 1998, $10.4 
million per year has been allocated to Idaho’s K-12 public school systems, an average of 
$42.55 per student, per year. 

In addition to providing monies for technology, the initiative provided for the creation of 
an administrative council to be established under the State Board of Education. The 
administrative council, named the Idaho Council for Technology in Learning (ICTL), created 
eight specific goals for technology in Idaho's schools by which to evaluate the impact of the 

State’s investment, and fulfill the Legislative charge. 

/ 

Legislative Charge to the ICTL 

The legislative charge to the Idaho Council on Technology in Learning (ICTL) is as 
follows: 



“The Idaho Council on Technology in Learning will develop statewide 
performance indicators to address the legislative intent of the impact, costs and 
benefits of the projects funded by the Idaho Council on Technology in Learning. 

The performance indicators will be used to track progress and funds spent to 
achieve performance indicators. A report of the funds expended to attain the 
performance indicators will be provided to the legislature and the State Board of 
Education annually." 

The intent language provided by the legislature specifically requires the ICTL to 
accomplish the following: 

1) Develop statewide performance indicators 1 

2) Address impact, costs, and benefits of projects funded through the ICTL 

3) Track progress and funds spent to achieve indicators 

The Data Collection Plan 

The design of the plan uses statewide tests, research studies, student surveys and 
technology examples to measure the effectiveness of technology in the Idaho’s public schools. 
Data collection was divided into three levels (tiers) and each level of data was used to answer 
a key question for the eight ICTL Goals 2 . The tiers are as follows: 

Tier One-An assessment which measures the relationship of technology on 
academic gain and can be generalized across the Statewide population of 




’See appendix (data collection summary) for individual performance indicators. 

2 ICTL goals five and eight were combined due to similarity In content and intent. ICTL goal six, evaluation, 
is being fulfilled by the completion of this study. 
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Idaho’s K-12 public school students. 

Tier Two-Reporting of regional technology studies which target specific learning 
outcomes. 

Tier Three-Reporting of descriptive examples supporting regional and statewide 
assessments. 

Effectively, the marriage of the three tiers of the data, via the collection design, 
provides “the big picture" ~ an ample blueprint of the effect of the investment in technology in 
education between the years 1994-98. 
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ICTL Answer* to Legislative Quastions 
Goal - Integration of Technology 



{Question 1 ~ Have students improved their academic performance as a result of the 
integration of technology in Idaho's K-12 schools? 

Answer i-A positive relationship between academic performance in core studies, 
language, math, and reading and the integration of technology can be shown in Idaho's 
K-12 schools. 



Cost - As reported by Idaho School Districts, the amount of ICTL monies expended to 
achieve ICTL Goal One, Integration in the 1997-98 school year, represents 36.26% of the | 
total ICTli dollars allocated. 



Evidence 

Tier One (student testing) - The question addresses if there is a relationship between 
technology and academic gain and if so, what it might resemble. 

The Methodology of the study follows: 

Subjects • u 

The population, from which the sample was drawn Includes approximately 244,400 K- 
12 public school students, in the State of Idaho, who are enrolled in the 1098-99 school year. 
The target sample population of 35,885 8 th and 11 th grade students, was comprised of 18,493 
males and 17,392 females. 

Groupings 

The subjects comprising the sample can be categorized into two distinct groups: 

Group One - Those who were in the 4 th grade in 1994 and who are currently in 8 th grade and 
will be taking the ITBS exam in October. 

Group Two - Those students who were In 8*' grade in 1995 and who are currently in 1 1 
grade and will be taking the Test of Academic Proficiency (TAP) in 1998. 

The 1994-95 and 1995-96 school years were used as the initial data points by which to 
track student progress over the four years in which monies were made available to districts 
through the ICTL. Two groups of students were chosen to include in the sample, the first to 
reflect participants who were enrolled as elementary students from 1994-98, and the second to 
reflect those participants who were enrolled as secondary students from 1995-98. Both groups 
were used to ensure the sample data collected would better represent the population than the 
use of only one secondary or elementary group. Additionally, the use of two groups takes into 
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account the varying disbursement of funding between and within the districts over the period of 
the study. For example, School District "A" may have determined that investing their monies in 
secondary student services was a stronger implementation rationale because high school 
students are closer to graduation and/or joining the workforce. School District "B" may have 
chosen to put monies at the elementary level first, and graduate towards the secondary level 
over time. By including two groups both integration strategies will be taken into consideration 
within the data collection model. 

Data Matchina/Elimination 

The data collection was limited to include students who were in Group One and 
matched by name and birth date between 1994 and 1998 school years, or in Group Two who 
matched between 1995 and 1998. The strategy was to eliminate any students who emigrated 
or immigrated during the period of study. This reduced the number of students in the sample. 

(n = 26,122) 

Instruments 

Few assessment tools are currently available in Idaho for use statewide. The following 
is a summary of the choice of assessment instruments and the rationale for such decisions. 
Criteria for choosing a statewide assessment instrument follow: 

Criterion 1 -- The measurement instrument had to be used statewide consistently 
between the years 1994 and 1998. 

Criterion 2 -- The measurement instrument could not have any major modifications 
during the years 1994-1998. 

Criterion 3 - The measurement instrument must have met the tests for reliability and 
validity. 

Criterion 4 -- The measurement instrument must have been designed to collect data 
which can be generalized statewide. 

As a result of the following analysis, the measurement instruments that were chosen for 
this study include: 

1) * ITBS exam scores for Idaho students in the 4 th grade in 1994 and 8 th grade in 

1995, as well as Iowa Test of Basic Skills (ITBS) exam scores for Idaho school 
students in the 8 th grade in 1998. 

2) Test of Achievement and Proficiency (TAP) exam scores for Idaho students in 
11 m grade in 1998. 

In addition to the measurement instruments, a ten-question self-reporting student 
survey was designed to measure technology exposure. 

The Iowa Test of Basic Skills (ITBS) Exams 

The ITBS is a nationally recognized standardized test of academic achievement 
consisting of a battery of assessments in vocabulary, reading comprehension, word study, 
language skills, mathematics, social studies and science. The test has been in existence for 63 
years and has a reliable track record (Hoover, 1994). The information from the battery is 
unique in the fact that it cannot be obtained from other sources (Hoover, 1994). The Iowa Test 
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of Basic Skills was developed in 1935 at the University of Iowa College of Education to help 
teachers determine knowledge levels of their students, for planning instructional goals and 
approaches, to provide achievement information to monitor student's progress year-to-year, to 
provide a basis for reporting to parents, and to identify areas of strength and weakness of 
groups as they relate to curriculum. Subsequently, the exam has been used for reports to 
administrators, principals and others for making decisions related to many aspects of the 
educational process, 

Norms 

The term “norm" refers to a set of scores that are used to make interpretations. National 
norms are scores from a nationally representative sample of students, while local norms 
originate from local school districts. The tests for the subject areas on the ITBS have been 
standardized with the same group of students, and have been obtained from a single group of 
students at each grade level. The use of norm groups generally allows the evaluator of the 
student data to make general statements about the student skill strength in tested areas, as 
well as to allow students to be compared to other students, and schools to other schools 
(Hoover, 1994 pp. 55-56). Idaho has changed the norm group used once since the 1993-94 
school year. The national norm group used over the period of study is the Fall 1992 group. 

The Test of Achievement and Proficiency (TAP) 

The 1992 edition of the Test of Achievement and Proficiency, used by Idaho, was 
designed to measure the ability of the student to use information, emphasizing critical thinking 
skills to a greater degree than previous editions. The survey battery includes the following 
sections: 1) Reading Vocabulary; 2) Reading Comprehension; 3) Written Expression; 4) Math 
Concepts and Problem Solving, and 5) Math Computation (optional) (Riverside, 1994). 

The ITBS/TAP assessments were chosen for the following reasons: 

First, to meet Criterion 1, the measurement instrument has been used consistently 
between the years 1994 and 1998. As seen in Table 1, the model which will be used is a four- 
point data comparison model. The critical years for data collection are 1994-95, 1995-96 and 
1997-98. There will be two base years (data points). The first data point 1994-95 will be the 
base on which students who were 4 lh graders in 1994-95 will be tracked through the second 
data point, 8 th grade in 1998-99. The 1995-96 year will be used as the third data point in which 
students who were in 8 th grade in 1995-96 will be tracked through the fourth data point, 11 th 
grade in 1998-99. 
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Table 1 



Pm- and Post-test Data Points for the Iowa Test otB 


asic Skills 




1 

! 


Data 

Point 


Pre- 

Year 


Pre-Year 

Grade 


Data 

Point 


Post- 

Year 


Post-Year 

Grade 


Group 1 


1 


1994-95 


4* Grade 


2 


1998-99 


8th Grade 


Group 2 j 


1 


1995-96 


8 th Grade 


2 


1998-99 


11* Grade 



There are two types of tests given in any year - the survey battery or the complete 
battery. As seen in Table 2, the survey battery was taken by students in the 4* grade in the 
1994-95 school year and the 8*' grade in the 1995-96 school year. Both the 8* and 11* grade 
students will take the survey battery in 1998, which will provide for our third and fourth points 
for data comparison. 
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Table 2 



Iowa Test of Basic Skills Descriptive Information 



iuna i uoi wi 

School Year 


1993-94 


1994-95 


1995-96 


1996-97 


1997-98 


1998-99 


t 

Norms ; 


Spring 88 


Fall 92 


Fall 92 


Fall 92 


Fall 92 


Fall 92 


Grades 1 
Tested , 


t 


6, 8,11 


4, 8, 11 


3-11 


3-11 


3-11 


3-11 


Dates Given 


March 


October 


October 


October 


October 


October 


Type of ! 
Test | 

» 


t 

1 

1 

! 

I 


Comp. 

Battery 


Survey 

Battery 


Grades 3, 
5, 6, 7, 9, 
10 took 
Complete 
battery. 
Grades 4, 
8, 11 took 
Survey. 


Grades 3, 
5, 7, 9 took 
Complete 
battery. 
Grades 4, 
6, 8, 10,11 
took 
Survey. 


Grades 3, 
5, 7, 9 took 
Complete 
battery. 
Grades 4, 
8, 8,10,11 
took 
Survey. 


Grades 3, 
5, 7.9 
took 

Complete 
battery. 
Grades 4, 
6, 8. 10, 
11 took 
Survey. 



Next, Criterion 2 was met as a result of the consistency of the measurement 
instrument. The instrument did not have any major revisions during the years 1994-1998, As 
seen in Table 2, the test examination date was standardized in the 1994-95 school year. The 
change to the October examination date gave enough continuity to the testing schedule that 
the data could be collected starting in the 94-95 school year for those students who currently 
are in the 8" grade and the 95-96 school year for those currently in the 11* grade. As seen in 
Table 3, there have been no significant modifications since 1994-95. 

Table 3 

Modifications in ITBS 1993-1999 — 

1995 - Change in grades tested (does not effect grades 4,8) 

1994 - Change in norms from Spring 1988 to Fall 1992 
1994 - Change in examination date from March to October 

1993-96 - Changes in exam types given to various grades 



Next Criterion 3 was met as a result of the measurement instrument meeting the tests 
for reliability’ and validity. The ITBS/TAP tests are a nationally recognized standardized 
achievement battery which has been used for over 60 years by many states to provide 
information to improve instruction (Hoover, 1994). The KR-20 (Kuder-Richardson) reliability of 
the Iowa Test of Basic Skills refers to the internal consistency test for reliability. As wen in 
Table 4, the reliability of the ITBS is .85 for grades K-8 for the Fall testing in years 1994-98 
(Riverside, 1994). 
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Table 4 

Reliability of Iowa Test of Basic Skills 1994-98 (based on 1992 Standardization) 

Reliability 

Instrument . Coefficient (KR-20) 

Grades K-3 Fall .85 

Grades 3-8 Fall .85 

As seen in Table 5, the KR-20 reliability of the TAP ranges from .829 to .950 depending 
on the subject. 

Table 5 

Reliability of Test of Achievement and Proficiency 1998-Standard Scores 
Reliability 

Instrument Coefficient (KR-20) 

Test of Achievement and 

Proficiency (TAP) .829-.950 . 

The State of Idaho uses the ITBS/TAP exams as valid measures of academic 
achievement. Content validity of the ITBS/TAP is ensured by Riverside Publishing through a 
rigorous set of development steps including: 1) developing content specifications: 2) editorial 
review, and 3) field testing (Riverside, 1994 pp. 11-14). 

Last, Criterion 4 was met as a result of the measurement instrument’s design. The 
ITBS/TAP exams are the standard set by the Idaho State Department of Education for the 
statewide evaluation of student performance (Personal Communication, Tiel 1998). 

The Student Technology Exposure Questionnaire 

A survey was developed to measure the technology exposure students had 
experienced over the four-year period under study. The procedure follows: 

1) Development of the survey 

2) Beta testing the survey 

3) Revision of the instrument 

The technology exposure questionnaire was developed using International Society for 
Technology in Education (ISTE) standards, a nationally recognized technology standard. ISTE 
recently released the National Education Technology Standards (NETS) for students (ISTE, 
1998). A group of Idaho technology experts was assembled, including regional technology 
advisors from colleges and universities, parents, teachers, State officials, and students to use 
the NETS objectives to create a self-reporting student exposure questionnaire which was used 
to measure the amount of perceived exposure to technology a student had between the years 
1994-1998. 

The questionnaire was then pilot tested on 8 th and 11 th grade classes and tested for 
statistical reliability using the Cronbach’s Alpha test, where (n=107). The results were used to 
choose the questions with appropriate content having the greatest reliability. Then, in October 
of 1998, the questionnaire was given to all Idaho 8 th and 1 1 th grade students who took the 
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ITBS/TAP exams. The student answers to the exposure questions were recorded in the 
student identification box on side one of the ITBS/TAP answer sheet. 

Riverside Publishing was contracted to deliver a CD Rom or series of diskettes which 
contained the data from all three instruments (Table 6). This data provided the responses of 
all students in the sample on each of the 10 questions in the exposure survey, archived data 
from ITBS exams for all Idaho students in 4 th and 8 th grade in the 1994-95 and 1995-96 school 
years respectively, and data from all ITBS/TAP exams for Idaho students in 8 th and 11 th grade 
in the 1997-98 school year. 

Table 6 

Riverside Publishing Reported Data 



Item . 

Grade (Based on post-test grade 11 or 8) 

System Name (District) 

Building Name (Based on post-test grade 11 or 8) 

Response to 10 survey questions 

ITBS/TAP Post-test Scores 
Reading Total SS 
Language Total SS 
Math Total SS 
Core Total SS 

ITBS/TAP Pre-test Scores 
Reading Total SS 
Language Total SS 
Math Total SS 
Core Total SS 

ITBS/TAP Gain Scores 
Reading Total SS 
Language Total SS 
Math Total SS 

Core Total SS 

The questionnaire data was imported into the Statistical Program for Social Sciences 
(SPSS) and the variables were plotted to determine the natural groupings. The data was 
analyzed and categorized into three groups, the first comprising the high exposure group, the 
second comprising the medium exposure group, and the last comprising the low exposure 
group. For example, students who answered the 10 technology exposure questions with a high 
degree of perceived exposure were classified into a “high exposure” group. Similarly, those 
students who perceived a medium and low exposure to technology respectively were 
classified. 
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Procedures 

The study seeks to determine: 

1) Whether students have increased their technological literacy as a result of 
exposure to technology in Idaho schools over the four-year period of study. 

2) Whether there is a relationship between exposure of students to technology 
and student academic achievement in Idaho schools over a four-year period. 

3) Which technology factors have the greatest impact on academic gain in 
Idaho’s schools. 

To make such a determination is a four-step process: 

1) Identify Academic Gain -- demonstrate student achievement using the 
Iowa Test of Basic Skills (ITBS) gain scores of two groups of Idaho 
students over the four years of study. 

2) Identify Technology Exposure -- identify the technology exposure of the 
students within this time period. 

3) Identify the Academic Gain and Exposure Relationship -- correlate 
student academic gain with technology exposure. 

4) Identify Technology Factors - identify which technology factors explain 
the greatest amount of academic gain. 

Data Analysis 

Analysis Model 

Multiple Analysis of Variance (MANOVA) 3 was conducted on the data to accomplish the 
following: 

1) Identify differences in ITBS/TAP scores between groups at grades 8 and 1 1 , 
and 

2) Compare each group’s ITBS/TAP core, language, math and reading scores to 
each other. 

The purpose of using the MANOVA within the research design is to help determine 
whether the observed differences in the samples could be attributed to the natural variability 



Multiple comparison procedures are designed to protect the researcher from calling 
differences significant when they really are not [Norusis, No Date #84] p. 291. 
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among the sample means or whether there was a reason to believe that some of the three 
exposure groups (high, medium, or low) had different values in the population with regards to 
academic gain. The following was tested: 

Hypothesis 

H 0 There are no significant differences in high, medium, and low exposure groups with 
respect to academic gain on ITBS/TAP core, reading, language, and math sections. 

H, There are significant differences in high, medium, and low exposure groups with 

respect to academic gain on ITBS/TAP core, reading, language, and math sections. 

By testing the hypothesis, we have determined the variability in the sample values. We 
studied how much the observations within each group varied as well as how much the group 
means varied. After the MANOVA (multi-variate analysis) was run for both the 8 th grade group 
and the 1 1 th grade group, an ANVOVA (uni-variate) was run on each area of the ITBS/TAP, 
reading, math and language 4 . Finally, a Tukey post-hoc analysis was run to assess the 
differences between the high/low, medium/low, high/medium groups. 

Independent Variables 

The High, Medium, and Low Technology Exposure Groups were used as the 
independent variables in this study. 

Dependent Variables 

The Math, Reading, Language, and Core ITBS/TAP subtests were used as the 
dependent variables. 

Results 

The following are the results of the analysis of Idaho student Core, Math, Reading and 
Language ITBS/TAP scores in relation to the three groups of students identified (high, medium 
and low exposure to technology). 



4 

A test used to determine if several independent population means are equal. 
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8 th Grade ITBS Effect Size Diff 


erence Low/Hiah Grouo Exposure bv Subject 


. i 


8* ITBS 
Core 


Q*' ITBS 
Reading 


8* ITBS 
Math 


8** ITBS 
Language 


Effect Size 


0.24 


0.23 


0.23 


0.19 


Months of Academic 
Increase ^4-years) 


2.40 


2.30 


2.30 


1.90 


f 

Months of Academic 
Increase p 2-years) 


7.20 


6.90 


6.90 


5.70 



As seen in Table 7, a comparison of students in the high and low exposure groups 
reveals that Idaho 8th grade students who reported experiencing high exposure to technology, 
experienced academic gains in core, reading, math and language ranging from .19 to .24 over 
those students who reported low exposure to technology. All groups showed that there were 
statistically significant differences between groups where F(8, 23906) * 14.13, at P <.001. 

Effect Size was calculated for each group comparison using the formula ES=MrMA 
where M represents the mean scores respectively of the high and low exposure groups, and 
summarizes the general degree of outcome between the two groups. Cohen (1988) reports in 
his work, Statistical Power Analysis for the Behavioral Sciences ( 2 " Ed) that the effect size R 2 
can be labeled by category for analysis purposes. When using MANOVA the following ranges 
apply: 

Small but significant effect = .2 SD separating the group means (r 2 =.01) 

Medium effect = .5 SD (r 2 = .059) 

Large effect * .8SD (^“.ISS) 

By using Cohen's work on meaningful effect sizes, we see that both 8*’ and 1 1 th grade 
Effect Sizes measure large enough to be considered as practical and meaningful. 

According to Rogers (1991), Glass, McGaw, & Smith (1981),. effect sizes which 
represent ability groupings can be extrapolated into months of academic gain. For example, 
the 8** grade core sample shows an ES * .24. This figure when converted shows an academic 
gain of 2.4 months of those students in the high exposure category as compared to those in 
the low exposure category over the four years of study. It can be inferred that for an 8th grade 
student who experiences high exposure to technology, over a 12-year school career, an 
academic gain of a little less than a school year (7 months) can be achieved through the 
integration of technology into the classroom. 
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Table 8 

II* 1 Grade ITBS/TAP Effect Size Difference Low Exposure Group/Hiah Exposure Group bv 
Subject 



i 


11* 

TAP Core 


11* 

TAP Reading 


11* 

TAP Math 


11* 

TAP' Language 


Effect Sl^a 


0.19 


0.20 


0.10 


0.10 


Months ojf Increase (3- 
Years) 


1.90 


2.00 


1.00 


1.00 


< ' 

Months of Academic 
Increase [{12-years) 


7.60 


8.00 


4.00 


4.00 

. 



The 1 1 th grade students who were reported in the high exposure group in the sample, 
similar to the 8 th grade, show academic gain in all four areas. Initially, the gains for the 11* 
grade group seem smaller; however, it must be noted that the period of the 11* grade study 
was only three years, while the 8* grade was four. For example, we find that over the three 
years of study students who reported high exposure to technology when compared to the low 
exposure group show 2 months of academic gain in reading. Again, it can be inferred that 
over the career of the student receiving high exposure to technology approximately 8 months 
can be gained in reading over a 12- year period. 

A Tukey post-hoc revealed that there were statistically significant differences where P< 
.001 between the low exposure and high exposure in the core, language, math, and reading 
groups. However, in the language and math groups no statistical significance was shown 
between the medium and high groups, and low and medium exposure groups respectively. 
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Tier Two (technology studies) 

In order to evaluate the relationship of integration between the teacher/student use of 
technology and student learning, state institutions of higher education undertook numerous 
studies using several public schools, different grade levels, and a variety of curricula. 
Completed studies were conducted using pre- and pcst-test assessments. There are, 
however, several studies still in progress at this time. The following is a synopsis of the 
studies: 



Table 9 

Boise State University Research Studies 



Study | 


Area Studied 


Type of Study 


Result 


BSU #1 


Sdem^uKimedia 


Quasi-Experimontal 


Positive Academic Gain 


BSU #2 


Vocabulary Development 


Experimental 


Positive Acadsmic Gain 


BSU #3 


Mathematic* 


Experimental 


No Significant Gain 


BSU #4 


At-Risk Students 


In Progress 


In Progress 


BSU #5 


At-Risk Students 


In Progress 


In Progress 


BSU #5 


Special Needs Students 


In Progress 


In Progress 


BSU #7 


Social Studies 


Experimental 


No SignMcant Academic 
Gain 


BSU #3 


Telecx>mmunk*tfon*MftRing 


In Prognats 


In Progress 


BSU #9 


Science/Databases 


In Progress 


In Progress 


BSU #10 


Writing 


in Progress 


In Progress 


BSU #11 


Technical Writing 


In Progress 


In Progress 


BSU *12 


Integration Vs. Lab Instruction 


In Progress 


In Progress 



(Contact Carolyn Thoraen) 

Number of studies ■ 12 
Numbar of studies completed « 4 
Number of studies in progress >8 
Studies showing positive academic gain * 2 
Studies showing no acadsmic gain » 2 

Example: 

Lead Researcher: Del Siegle , , A . . 

The Impact of Presentation Software on Secondary Science Students Achievement 

and Attitudes (Glenn’s Ferry High School) 
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Two groups (A&B) of Anatomy and Physiology students were involved in a study 
using laptop computers. The computers were made available to Group A during the first half 
of the year while Group B did not have access. For the second half of the year the procedure 
was reversed with Group B having access to the computers while Group A was without. 
Continuity was maintained between the two classes by using the same teacher, Identical 
curriculum, and teacher-generated tests to measure student achievement. Interestingly, after 
Groups A and B rotated, the students in Group B made up the deficit in test scores as 
compared to Group A, Differences between the groups were clearly evident. Students using 
computers scored an average of a full grade (i.e. A to B) higher than students not using them. 



Table 10 

riaho State University Research Studies 



Study j 


Arta StudloS 


Type of Study 


Result 


ISU #1 


Moth/Pf oWom SoMoq 


In Progress 


In Progress 


ISU *2 


Geography Database 


In Progress 


In Progress 


ISU #3 


Math 


Quasl-Experimantal 


Positive Academic Gain 


ISU #4 


Teaching Styfas 


In Progress 


in Progress 


ISU #5 


Teacher Attitudes 


In Progress 


In Progress 



(Contact At Strickland) 

Number of studies reported* 5 
Number of studies reported completed * 1 
Number of etudiee In reported in progress * 4 
Studies reported showing positive academic gain * 1 
Studies repotted showing no acadsmic gain * 0 

Example: 

Lead Researcher - Dr. Ai Strickland/Dr. J. Coffland 
ISU Math Study 

The aim of the ISU math study was to determine if the use of diagnostic teaching, 
math manipulatives, and computer-assisted instruction, properly used, would Improve the 
achievement of 5 th grade students in math. The study involved 22 fourth-grade teachers from 
21 classrooms in nine Southeast Idaho schools. Teachers were given year-long training while 
using math manipulatives, five computers per classroom, and appropriate math software for 4 
grade math instruction. Student gain over the year was measured with the Stanford Diagnostic 
test - which showed an average student gain in math concepts of 16 percentile points, and the 
ITBS test - which showed an average gain of 18.9 percentile points between students' fourth 
and fifth grade scores. 
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Table 11 



University of Idaho/ 


_ewis and Clark State Colleae Research Studies 


Study | 


Ana Studied 


Typ# of Study 


R#«u* 


Moscow #1 


Intsmat 


ExperimaoUl 


No Statistical Slgnftcanca 


Moscow #2 


EnhsnoamsntAkit 


Quas^£xpsdmsnta) 


PoaWva DNferanca 


Moscow #3 


Mathamatics/Hyparmadla 


Quas^&cperimsntai 


Poattre Difference 


Moscow #4 


Witting Skin* 


In Prograts 


In Prograss 


Moscow #5 


DMktop Vtd*o Conferencing 


Deschptlvt 


Posftfc* 


Moscow #9 


Docktop Vkfeo Conferencing 


In Prograst 


In Prograss 


Moscow #7 


Intsmet 


In Prograst 


InProgrtss 


CDA«1 


Tsachtr Computer efficacy 


In Prograst 


InProgrtss 


CPA #2 


Toactor Dovctopmont 


In Progress 


In Prograas 


CPA #3 


Intsgration 


In Prograts 


InProgrtss 


COA#4 


Mathematics 


In Prograss 


InProgrtss 


CDA#S 


Mathematics 


InProgrtss 


In Prograas 


CDA#6 


Professional Dsvttoptwrt 


In Prograss 


In Progrtss 



(Contact Dr. John Davlt/Dr. Haidl Rogara) 
Number of studies >13 
Number of studies completed * 3 
Number of studies In progress >10 
Studies showing positive academic gain - 2 
Studies showing no academic gains * 1 



Example: 



Investigating the Use of Microcomputers as Enhancement Aids to Facilitate 
Lessons by Teachers 



The 53 elementary students participating in the study were divided into experimental 
and control groups, and similar instruction was delivered to both groups. Following instruction, 
students in the experimental group used appropriate software for practice activities, while 
those in the control group used traditional worksheets and workbook activities as enhancement 
aids. The Iowa Test of Basic Skills was used to gather pre- and post-test data. Statistical 
analysis favored computers as effective enhancement aids to the content areas of 
mathematics and language arts. 

As seen above, of the 30 research studies initiated by the institutions of higher 
education, 8 studies were completed and 22 are still in progress. Results of the studies were 
mixed - of the 8 studies that were completed 5 show positive academic gains as a result of the 
integration of technology, while 3 studies show no significant gains. Positive gains were found 
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in the areas of math, video conferencing, enhancement aids, science and vocabulary 
development. No significant academic gains were found In the studios targeting social studies, 
mathematics, and Internet. It is important to note that the results rely heavily on the type, 
controls, and methodology of each study. 

Some Idaho school districts chose to conduct studies on the effects of technology in 
learning locally. Thirty studies were attempted, ranging in type from experimental to survey. 
Although many of the studies had questionable methodology and controls, overall positive 
results were found in the studies which were reported as complete. 



Table 12 



District | 


Area Studied 


Type of Study 


Reset! 


Bote 


Reading 


Quasi- Experimental 


Positive 


Hsnten 


Math. Lang Aits, Reading 


inProgres* 


In Ptogrow 


Shoshone 


Reading 


Quari£xperimental 


Positive 


Valley 


Math 


Descriptive 


Positive 



Contact (ICTL Staff-State Department of Education) 
Number of studies reported ■ 30 
Number of studies reported complete ■ 18 
Number of studies reported in progress >1 1 



Boise School District #1 

Using the Waterford Early Reading Program. 155 students in two schools were 
Involved In a study designed to evaluate the impact of specific software on the learning 
process. Control and experimental groups were used and pre- and post-tests were 
administered. The results of the study indicated that the group using the Waterford program 
outperformed the control group. 



Shoshone School District #31 2 

Students in grades 8 and 12 were used in this study to determine if technology had an 
effect on reading achievement scores. The STAR reading assessment test was used as a pre- 
and post-test measurement Instrument. Students working with programs enhanced by 
technology demonstrated overall improvement of 1 .05 grade levels in one school year. 
Students in grades 12 demonstrated an Increase of 2.5 grade levels. ESL students showed 
steady improvement of at least a full grade and as much as 2.5 grade levels. 

Additional examples are located in the Appendix. 
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Tier Three, (technology exampiet) 

Included in this section are descriptive examples gleaned from the ICTL Phase Two 
Reports completed by school districts yearly. The examples focus on the benefits derived from 
the integration of technology and how it has affected student performance in the classroom. 

Benefit Example One: Effect on Test Scores- Many school districts were interested In 
determining how technology had impacted various test scores over the past year. Nampa, 
Filer, Payette, and New Meadows reported examples of increased ITBS scores as a result of 
integrating technology. Additionally, as seen in Table 13, fourteen districts reported 
substantial increases in reading scores or a dramatic increase in library circulation as a result 
of the integration of software specific programs such as the Accelerated Reader program. 



Table 13 



dtcu rvuougi nouuirou w- 

District j 


Increase In Reading Secret 


Increase In Library 
Circulation 


American Falla 


.75 grade levola In four months 


200% 


Bonnier 


N/A 


225% 


Buhl 


N/A 


Triptod 


Burtay 


20% 


N/A 


Cottonwood 


S month* in a 6 weak partod 


200% 


MlddMon 


N/A 


22,060 Book* 


North Oam 


.45 gcado kvtte in 2.3 months 


1,122% 


Prttton 


N/A 


600% 


Rkto 


t grad# lavaline months 


100%. 




N/A 


12,300 


SwunVdUy 


1 gradotavtJ 


200% 


WMt Jefftreon 


Raportad Incraata 


| 120% 

akiit* lannirartcs art* /Snda Sorinas. 



Grangavllle, Castlafbrd), and ACT acorea (Idaho Falla) 

Benefit Example Two: Cooperative Learning- Students in the Kuna School District s 
Elementary/Secondary writing program ‘Writing Partners' use technology to work 



* Murtaugh, Buhl, American Falla, Sugared am. LakaUmd, v*at *^, **•• • 

MMdiaton, Swan VaWty and Bonner have reported lncree*ae In either wading scone, iwuy circulation, andTor atudara uma on raw as 

result of the use of the AeoslaratadRaedar program. 
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collaboratively with other students on language arts projects. This program pairs 4 th , 6 th and 8 lh 
grade students with counterparts in the 1 1 th and 12 th grades. These students met face-to-face 
a couple of times at the beginning of the school year and then continued to collaborate on 
class projects over the course of the year using telecommunications to conduct peer editing 
online. Teachers have reported increased interest, motivation and quality in the students 
writing. More information is available on the district website: www kunaschools.org/writing.htm 

Additional examples are provided in the Appendix. 
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/ ■ ■ ■ T 

Question 2 - Have Students improved their technological literacy as a result of exposure 
to technolog/? 

Answer! Both 8* and l'l‘ h gfade groups increased their perceived technological literacy 
as a result of exposure to technology. 



In October of 1998, students were asked to complete a technology exposure survey 
prior to taking the ITBS or TAP exams. The results of the survey revealed the amount of 
perceived exposure to technology the students had over the four years of study. As a result of 
their responses the students were categorized into three groups, high, medium and low 
technology exposure. The data from both 8 th and 1 1 th grade groups show that the majority of 

students reported moderate exposure to 

_ f „ ^ technology over the four years of study. The 

Fig. 1 : 1 1th Grade Exposure results of the survey are as follows: 




As seen in Figure 1, out of 12,494 valid 
cases, 2,539 or 20.3% of the 1 1* grade students 
reported that they had experienced a low 
exposure to technology, while 7,892 or 63.2% of 
students reported a moderate exposure. Those 
students reporting a high exposure to technology 
represented 2,063 or 16.5% of students 
responding. 



| | Lew [3 Medium 
□ High 
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Fig. 2: 8th Grade Exposure 




As seen in Fig. 2, out of 1 1 ,554 vaiid 
cases 2,176 or 18.8% of 8 th grade students 
reported experiencing a low exposure to 
technology over the four years of study, while 
7,367 or 63.8% of the students reported a 
moderate exposure. Those students 
reporting a high exposure to technology over 
the four years of study accounted for 2,01 1 or 
17.4% of students reporting. 



[HI Low H Medium 
□ High 



The data clearly shows that, although there are many students who still have not had 
the satisfactory exposure to technology, the majority of Idaho students have experienced 
moderate to high exposure to technology. 
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Question 3 - What technology factors have the greatest impact on academic gain in 



\ . 



Idaho’s schools? 

Answer -The combined consistent high loading factors for 8* and 11* grade students 

were: i ^ ,* 

The ability of the stu'dent to choose the appropriate software tool for 

completing a project; 

Amount of computer use at school; 

Exposure tdfntemet and email use, and 
Amount of computer use at home. 



A stepwise regression analysis was conducted on both 8* and 1 1** grade groups, using 
gain scores for math, reading, core and language as the dependent variables and the 10 
questions in the technology exposure survey as the independent variables. A regression 
formula was used 



Where: 

Y= Gain Score Core, Gain Score Math, Gain Score Language or Gain Score Reading 
and X,...X 10 = Technology Exposure Survey Questions. 

The resuits of the regression reveal that the ability of the student to choose the 
appropriate software tool was consistently the highest loading variable in the regression, 
showing that it explains the greatest variance in the IT8S/TAP gain scores over the four years 
of study. 

Second only to the ability to choose the appropriate software tool was the amount of 
computer use in school. One can infer that access to computers by students has a strong 
relationship to academic achievement. Interestingly, the amount of computer use at home also 
loaded consistently high, showing a relationship to academic achievement. Issues of access 
to computers at school and home should be a priority to educators as well as parents. 

Two variables loaded high in the 11** grade regression analysis that did not show up in 
the 8 th grade regression analysis: use of software to simulate real-world experiences and 
working together in groups, using technology to complete class projects and assignments. 
These two factors seem to be good indicators of academic gain in 11* grade, while not as 
prevalent in the 8* grade. It could be hypothesized that the difference may be either the result 
of differences in curricula at the various levels, style of teaching or the difference in maturity 
between the 8 th and 11* grade groups. 
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Goal - Compatibility 

The performance indicators chosen to measure the extent to which the school districts 
have been able to share information and resources through the use of compatible equipment 
are: 

1) Number of local area networks; 

2) Number of wide area networks; 

3) Email use; 

4) Internet use, and 

5) Distance Learning use. 



uluastiop - Have schools been able to share information and resources through the use 

of , 
technology? 

Answer* Results from the school districts revealed extensive use of Internet and email by 
both teachers and students irt addition to moderate use of Idaho's Distance Learning 
facilities.. 



('Cost- As reported by Idaho school districts, the amount of ICTL monies expended to 
achieve I^TL Goal Compatibility in the 1997-98 school year represents 13.92% of total 
I^CTL dollars allocated. 



Evidence 
Table 14 



Item Surveyed 


Participants 


Students using district-provided email 


66,461 


Students using district-provided Internet 


191,738 


Teachers using district-provided email 


12,960 


Teachers using district-provided Internet 


18,107 


Distance Learning university to school 


315 


Distance Learning school to school 


380 



‘Note: The Idaho K-12 student population Is approximately 244,400 for school year 1998-99. There are 
approximately 15,000 certified teachers in the State of Idaho (Source Idaho State Department of Education). 
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Fig. 3: Internet Use 




Internet Um 

HI Idaho | Nation 



While the Internet is not a panacea for 
education’s woes, it does provide immense resources to 
Idaho’s schools which otherwise would remain 
untapped. The Internet provides rural as well as urban 
schools with the opportunity to go beyond merely being 
consumers of information. The Internet provides 
Idaho’s students with the ability to become information 
producers and communicators through collaborative 
instruction, inter- and intra-school. 



Fig. 4: Idaho Internet Use 1997-199B 




Internet Uae 

[H 1967-98 ■ 1996-97 



According to the the latest Market Data Retrieval 
(MDR) report, Internet access nationwide has increased 
dramatically; 85 % of schools now have Internet access 
compared to 70 % in 1997 and 32 % in 1996 (MDR, 
1998). Idaho, in comparison, has increased Internet 
access 1 3% over the past year to 96% of schools, and 
ranks fourth in the nation in access, preceded only by 
Maine, Tennessee, and North Dakota (QED, 1998; MDR, 
1998, p. 65). 



Fig. 5: Comparison of LANs and WANs 




LANs WANs 



As seen in Figure 5, connectivity within and 
among schools nationwide has significantly improved. 
Over 75% of schools reported using a LAN in 1998, 
while 44% of schools reported using a WAN (MDR, 
1998). Again, Idaho reports above the nation in 
comparison, showing that 88% of school districts have 
LANs installed, approximately 1.9 per district, while 61% 
reported that they use a WAN to communicate (QED, 
1998). 



£3 Idaho ■ Nation 
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Fig. 6: Student Email Access 




IH Access H No Access 



students with access was reported at 



A quick comparison of QED to MDR data 
reveals that Idaho ranks ninth in the nation for schools 
using WANs (QED, 1998; MDR, 1998, p. 54). 

As seen in Figures 6, 7 and 8, email use by 
teachers and students, non-existent only a few years 
ago, has made a tremendous impact in our schools. 
Nationally, 40% of teachers reported not having 
access to email (MDR, 1998). In comparison, 
approximately 84% of Idaho teachers reported having 
access to district-provided email, while the number of 
approximately 30% (ICTL Phase 2, 1998). 



Fig. 7: District Provided Email Access fo€ 



[ 2 . 0 »| 




Q Accmi H No Access 



Fig. 8: Email Access Comparison Idaho and€ 




13 Idaho OH Nation 



Fig. 9: Distance Learning Use 




94-95 95-96 96-97 97-96 

E3 Students served by distance learning 



The analysis of the final performance 
indicator for compatibility, Distance Learning use 
shows that, with the addition of Federal monies 
to existing state dollars, a marked rise in 
Distance Learning can be seen between the 
1994-95 school years and the 1997-98 school 
years. 
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Goal - Teacher Preparation 

One of the most pertinent issues surrounding the integration of technology into existing 
curricula, nationally as well as in Idaho, is teacher preparation and staff development. The 
colleges of education and school districts in Idaho have instrumental roles in preservice 
education, carrying the responsibility of providing professional development for the teachers in 
the State of Idaho. The intent of ICTL Goal 3, Collaboration with Higher Education, is to help 
facilitate relationships between the colleges of education and school districts in these regards. 
The performance indicators chosen to measure the provision of preservice to school districts 
were* 

1) Performance of Idaho Teachers on the Teacher Technology Certification; 

2) Number of workshops offered to school districts by higher education institutions; 

3) Total number of hours of instruction provided by each institution; 

4) Number of inservice hours provided by school districts, and 

5) Descriptions of exemplary teacher preparation. 



'Question - Have schools and colleges of education worked together to effectively 
prepare instructors to teach using technology? 

i * ■* *■ 

Answer - Schools and colleges of education have been working together to effectively 
prepare instructors to teach using technology. 



("Cost- As (reported by Idaho School Districts, the amount of ICTL monies expended to 
achieve ICTL Goal 3, Collaboration with Higher Education; in the 1997-98 school year 
Represent* 3.42% of the total ICTL grant dollars allocated. 



Evidence - 
Table 15 

Performance of Idaho Teachers on Teacher Technology Certification 7 



Institution 1 

i 


Tsksn 

Instrument 


Passed 

Instrument 


Tsksn 

Projsctod 1990 


Tsksn 

Projsctad Total 


Projsctod % 
Pass Rats 


BSU 


4,955 


3,418 


3,000 


7,955 


69% 


ISU 


30 


30 


76 


106 


100% 


Uofl/LCSC 


137 


105 


1,450 


1,587 


77% 


Totals 


5,122 


3,553 


4,526 


9,646 





7 Th*r* are approximately 15,000 owtifisd tMctwra In ktaho. Thsss numb#** do not Include pflsssrvtos Isschtrs 
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Table 16 



flOUlUUUII moviviuo waiq 

Institution i 


Workshops 


Hours 


BSU 


203 


3,114 


ISU 


68 


1,020 


LCSC 


43 


872 


Uofl 


77 


468 


Totals 


391 


5,474 




As seen in Table 16, Idaho institutions of 
Fig. 10: Comparison of Professional Training higher education have conducted 5,474 hours of 

training. An estimate can be made on the total 
number of training hours needed by multiplying 
the number of hours needed for pre-service 
teachers to certify (90) by the number of 
teachers certified in the state(1 5,000). To 
satisfy requirements needed to pass the 
assessments, Idaho teachers would need 
1,350,000 hours of training assuming that no 
training had been conducted. We know for a 
fact that training is taking place through 
Institutions of higher education as well as local 
school districts. 

□ Idaho Q Nation 

Nationally, 36% of schools offer no 

technology-related professional development (MDR, 1998, p. 109). As shown by district ICTL 
reports, many Idaho schools have conducted in-district inservice classes using their own 
training experts and facilities. The practice of providing the training in-district makes it difficult 
to quantify the actual time and resources a district allocates toward training of their teachers 
and staff. To help alleviate the problem, the ICTL asked the districts to report their 
contributions toward training by hour and individual for the 1997-98 school year. Idaho s 
school districts reported a total Of 11, 942 teachers who participated in technology training for 
a total of 86,603 hours of training that could not be directly related to out-sourcing expenses. 
As seen in Figure 10, Idaho school districts reported on the whole that 77% of staff received 
some professional-technical training, while the national average is 64% (ICTL Phase 2, 1998; 
MDR, 1998, p. 109). 
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Descriptive Examples of Successful Teacher Inservice: 

Example 1, Train-the-Trainer 

(as reported by Donna Vakili, Boise State University) 

A good example of technology-related training is the BSU Train-the-Trainer program. This 
program is an innovative collaboration between the BSU Educational Technology Outreach 
Program, an individual school district, and a teacher identified by the district as a potential 
technology trainer. It evolved two years ago as a way to expand the number of teachers 
trained by the BSU Educational Technology Outreach Program to meet the State-mandated 
challenge to train all teachers in Region 3, and to serve individual district needs. As the 
demands for effective technology training increase, it has become evident that these trainers 
play an important role in offering technology classes that meet the needs of educators. These 
trainers serve as technology advocates and integration specialists in their classrooms and can 
offer classes in educational technology that meet district goals and the needs of teachers. Not 
only do these trainers share a common knowledge of technology, but they also have 
developed a common vision for appropriate use of computers in the teaching/learning process. 

The Train-the-Trainer Program is a field-based, two-year program which focuses on 
technology skills, integration strategies, adult learning, and methods for teaching technology 
classes. Classes are held in a school district in a 12-credit sequence and can be applied to a 
Master’s Degree in Educational Technology or Curriculum & Instruction. A key component in the 
program is consistent follow-up and support by Boise State University to insure that the trainers 
do not feel isolated as they struggle to implement technology in their classroom and share these 
successes with others. 

There are currently 1 24 trainers in Boise State University’s T rain-the-T rainer Program . They 
represent 1 8 school districts: Bruneau-Grandview, Emmett, Homedale, Idaho City, Kuna, Marsing, 
Meridian, Middleton, Midvale, Nampa, New Plymouth, Notus, Parma, Payette, Twin Falls, Vallivue, 
Weiser, and Wilder. These trainers have trained 1,299 teachers through either formal workshops 
or technology inservices from Fall ‘97 through Summer ‘98. It is anticipated that these trainers will 
impact over 1,700 teachers in the coming school year. 

Example 2, Technology Inservice Project (University of Idaho-Moscow) 

A workshop was held for elementary teachers at the Nez Perce School District; that 
focused on spreadsheet application. The goal of the workshop was to increase participants’ 
skills, enabling them to begin to integrate spreadsheets into classroom instruction. The 
workshop focused on a specialized activity titled, "What's in the Bags: M&Ms." We have found 
that using this engaging and enjoyable activity is a great way to introduce teachers to 
spreadsheets. They attended to four tasks: get started with spreadsheets, find the average 
number of M&Ms in a bag, make an inference about the colors (quantity, percentage, etc.) in 
an M&M bag, and graph the results of their inquiry. As a result, the teachers (without being 
told) addressed the following specific objectives: 

• Learn how to enter information (data) onto a spreadsheet template 

• Create fundamental (arithmetic) calculation formulas and apply to data 
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• Modify and enhance cell entries with formatting styles 

• Arrive at inferences concerning available data 

• Graph results obtained from manipulated data 

Teachers responded positively and enthusiastically to the workshop. They reported that 
they not only understood how to use the program, but also learned function application of the 
program in their classroom. Many indicated that, with limited additional effort, they would be 
able to teach their students to use spreadsheets to begin to manage, manipulate and analyze 
data using the M&M activity. Follow-up visits by the inservice trainer confirmed the 
effectiveness of the workshop, as many teachers had already begun to integrate spreadsheets 
into classroom instruction, and to use it as a data management tool. This is an example of 
many successful workshops coordinated by the College of Education, University of Idaho. 

Example 3, Technology Inservice Project (University of Idaho-CDA) 

A successful example of integrating multimedia into the elementary classroom took 
place in the Coeur d’Alene School District. A six-month series of multimedia training workshops 
was delivered to 12 elementary teachers. The sessions took place once a month on a selected 
Tuesday. Teachers were released from class for training from 8:00 a.m. until shortly after 3:00 
p.m. The training sessions took place in four different elementary schools using different labs 
and equipment during each session. Final projects were saved and transferred to CD-ROM. 

Participating teachers explored a different element of multimedia development during 
each of the first five months of training and were challenged to integrate the concepts into a 
final project. Training included use of equipment and peripherals, along with specific 
multimedia-development software training. Teachers explored the elements of digital audio, 
digital video, digital imaging, text elements, and interactivity in a variety of activities designed 
to introduce the basic skills necessary to begin a successful project. The Internet and its 
potential for contributing multimedia resources was also presented and explored. The teachers 
learned to search for graphics, text materials, audio samples and video clips to incorporate into 
their projects. 

The final class projects were an impressive display of creativity and ingenuity. The 
projects were displayed at the Coeur d’Alene mall to shoppers taking the time to stop and 
interact with teachers and students about their projects. One of the viewers was the State 
Controller; he expressed amazement at the quality of the projects and the manner in which 
they were displayed at the mall. Many students and parents visited the mall to see the 
displayed projects. This multimedia training project proved to be a great success for the 
teachers and students involved, and serves as one of the models for effective educational 
technology training coordinated by the College of Education, University of Idaho. 




29 



228 



000000001676 



Idaho Technology Initiative • Accountability Report 



Goal - Collaboration with Communities and Businesses 

The intent of the ICTL Goal 4, Community Collaboration, is to encourage local districts 
to develop relationships with their community patrons, businesses, and postsecondary 
institutions, involving them in a collaborative educational process. Performance indicators 
chosen to measure the amount of interaction are: 

1) Monies generated through local technology bonds; 

2) Monies generated through local technology levies; 

3) Monies generated through grants, and 

4) Monies generated through “other* donations. 



Question - Have schools involved community members, businesses, and 
poStseco'ndary institutions in the implementation and use of technology in 
schools? 

I \ i 'Y 

Answer - Community members, businesses, and postsecondary institutions have 
been involved in the implementation and use of technology In schools. 



/'Cost - Ae reported by Idaho school districts, the ICTL monies expended to achieve ICTL 
Goal, Collaboration with Community, Business and Postsecondary Institutions, in the 1997- 
98 school year represent 2.1 1 % of the total ICTL dollars allocated. 



Evidence • 

Each year since 1994, the ICTL has 

Rq. 1 1 ; Technology C onttixiti ons disbursed $10.4 million to Idaho school districts for 

installation and integration of technology in the K- 
12 public school system. 



As seen in Fig. 1 1 , 48.9% of contributions 
towards technology in 1997-98 originated from 
local districts. ICTL funding accounted for 34% 
while the remainder was made up of private, 
federal and other dollars. 

□ ICTL S3 

[3 Prtvata ■ Local 
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Table 17 



District Contributions to Techno 


OdV 1996-98 


TVp* | 


1996-97 


1997-98 


Federal Grants 


$1,467,555 


$3,093,346 


Private Grants 


$328,650 


$2,393,270 


Local Technology Bonds 


$201,356 


$131,250 


Local Technology Levies 


$3,283,313 


$3,228,849 


Local Tech Budget Provision 


$9,663,073 


$11,453,403 


Other (Donations. PUC, etc.) 


$5,630,296 


$2,071,946 


Total* 


$20,674,243 


$22,372,064 



Table17 shows that as a result of ICTL funding, Idaho school districts have had the 
opportunity to involve patrons, local businesses and industry through bonds, levies and 
donations. As seen in Figure 11, the school districts have at least matched the ICTL 
contribution dollar for dollar. 

Example, Integration and Job Opportunities* 

Through support from the Division of Vocational Education as well as funding through 
ICTL, many school districts have implemented Technology Support Technician Programs. 
These programs are designed to integrate the concepts of computer repair and network 
management into an industry-certified curriculum. Many school districts have realized the 
benefits of such a program by using their students to help install, troubleshoot and maintain 
the district's networks. Additionally, many of the students, after the first semester, have ( 
enjoyed paid work experiences either in the schools or in their surrounding communities 
businesses. Wallace School District is an example of a district which has used the program to 
benefit the students, school and surrounding communities. 

A cooperative agreement between Wallace and Mullan school districts and a local 
cable provider allows the students to run their own television station (Channel 3). The 
participating students gain valuable skills, which will serve them well in their adult lives as well 
as provide a needed service in their community. 
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Goal ^Technology Systems Enhancing the Efficient Operation of Idaho Schools 

Today’s technology systems allow speedy, accurate access and storage of 
Information, for decision making and reporting of student records to parents, school 
administration, and the State Department of Education. With the creation of efficient 
technology systems, Idaho’s schools can function in an effective manner. The performance 
indicators chosen to measure efficiency include: 

1 ) number of schools keeping track of student data electronically; 

2) number of networked computers used for administrative purposes, and 

3) number of districts submitting electronic reports to the State Department of 
Education. 



Question - Have schools used technology to improve administrative efficiency in school 
operations? 

Answer r Schools have used technology to improve administrative efficiency in school 
operations. 



Cost * As reported by Idaho school districts, the amount of ICTL monies expended to 
achieve ICTL Goal 5, Technology Enhancing Efficient Operation of Schools; In the 1997-98 
school year represents 39.48 % of the total ICTL dollars allocated. 



Evidence • 

The number of networked computers in Idaho schools used for administrative 
purposes, was reported by building. According to QED, there are currently an average of 73 
computers per district as well as 9.3 per district connected to the Internet (QED, 1998). 

As seen below in Figures 12, 13, and 14, many school districts have migrated from 
manual systems of accounting for student data to an updated electronic media. Data from the 
1997-98 ICTL Report reveals that of the 112 districts, 93% track student attendance 
electronically, 66% track discipline electronically, 63% track meals electronically, and 88% 
track grades electronically (ICTL, 1998). 
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Fig. 12: Discipline Tracking 




An additional benefit for Idaho’s districts is the ability 
to submit reports online or in an electronic format. In the 
1997-98 school year, 641 of 665 sites submitted 
accreditation information to the State Department of 
Education online. Moreover, 100% of school districts 
submitted enrollment count, attendance count, IBEDS, 
IFARMS and Special Education reports via diskette. 



171 Electronic H Manual 



Fig. 13: Grads Tracking 




O Electronic | Manual 

Example- 



Fig. 14: Attsndanca Tracking 




|j3 Electronic M Manual 

Descriptive 



An example of a technology system enhancing efficient school operation is the Kuna 
School District web site. Installed in 1997-98, the web site gives all students, administrators, 
and staff access to important information such as bus schedules, lunch schedules, yearly and 
monthly calendars, and extra-curricular schedules. Parents with Internet access can use the 
web site to find e-mail addresses for staff, find schedules for athletic events, and find specific 
information about their child’s school. Teachers use the web site to find links to other 
educational resources, find information on professional development opportunities, and 
register for in-district technology classes. Some teachers have posted classroom information, 
such as class rules, policies, and daily homework assignments so both students and parents 
are kept up to date on classroom activities. High school science students take quizzes on-line, 
and receive immediate feedback on their answers. Examples of increased efficiency include: 
1) reduced paper usage; 2) reduced number of parent telephone calls, and 3) quick 
dissemination of information to staff, parents, and students. 
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Goal - Training of Students to Maintain Technology 



Over the past four years, school districts have been afforded and unprecedented 
opportunity to install, configure, and maintain technology systems. Allowing students to learn 
from this process, along side teachers and administrators, can be a cost efficient means of 
deploying and maintaining these technologies. Additionally, the experience provides students 
the opportunity to acquire a highly sought after skill set. The performance Indicators chosen to 
measure the opportunity students have had to install, maintain, and support technology 
include: 

1) the number of Technology Student Technician (TST) programs in the State of 
Idaho, and 

2) the number of enrollments in the TST programs. 



Question - Are students able to Install, maintain, and support technology? 

Answer • Students have been given the opportunity to install, maintain, and support 
technology. 



'Cost -As* reported by Idaho’s sctibol districts, the ICTL monies expended' to achieve 
ICTL Goal 7, Student Training: In the 1997-98 school year represent 3.38% of the total 
JCTL dollars allocated. 



Pfe Mumkar «T TIT 




□ f w oiiw 



Evidence • 

Currently, there are 42 TST programs in 
operation in Idaho’s secondary schools, with an 
enrollment of 1,221 students. Recently, through a 
private grant program (The Technical Network for 
Training) has been set up to bolster the amount and 
quality of TST programs in Idaho’s secondary schools. 



Fta.1fcTSTEnraKnwnta 




— I i l 

FYM FY *7 FtU 



As seen in Figure 16, the number of TST 
programs has risen from 11 In 1996 to 42 in 1998, with 
a substantial increase in enrollment from 144 in 1996 
to 1,221 in 1998. 



□ EmUmntm 
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Summary and Recommendations 

This report clarifies issues surrounding the debate over whether the use of technology 
in Idaho’s schools has had a positive impact on students. The benefits of technology in 
teaching and learning are clear: an increase in academic achievement in reading, 
mathematics, language and core studies, improved technology literacy, increased 
communication, well-trained, innovative teaching, positive relationships with the community, 
more efficient operation of schools, and technically qualified students ready to enter today’s 
workforce 

By appropriating funds, through the Idaho Technology Initiative, the legislature has 
made a valuable investment in Idaho’s future which has paid and will continue to pay great 
dividends as shown in comparisons between Idaho and the Nation. Moreover, the funds 
distributed to districts, thus far have made it possible for Idaho schools to create a base upon 
which to build technology infrastructures such as hardware and software, compatible networks, 
timely teacher and student training, all which are key to providing for our students’ needs in the 
21 st century. 

It is important to note that schools, and local communities, as well as business and 
industry, have shown their support for the initiative by contributing monies for technology, 
extending the benefits of the initiative funds. However, to continue to experience these 
benefits, Idaho schools require continued support, not only from their communities and private 
sources, but from the state. Equipment must be upgraded and repaired to keep in step with 
industry standards. Teachers must continue training to be able to use the latest effective 
integration strategies in the classroom. Idaho’s students deserve and require training on 
cutting-edge, state-of-the-art technologies to be able to compete with their counterparts in 
higher education as well as in the global marketplace. 

Based on the gains shown in the first five years, it is recommended: 

• The Idaho Legislature continue to fund the Idaho Technology Initiative for an additional 
five years. 

• The Idaho Council for Technology in Learning (ICTL), continue to act as a governing 
body for Idaho school districts, by: 

- continuing to collect and report quantitative data on the benefit of technology 
for Idaho’s students. 

- assisting districts to refine and expand the use of technology integration in the 
teaching and learning process. 

- maintaining school district and higher education accountability. 

- teacher training for integrating technology is of primary importance and should 
be funded appropriately. 

- providing state-of-the-art technology, meeting requirements of higher 
education and business. 

- training on-site technicians to support the infrastructure that is already in place. 

• The Idaho Council for Technology in Learning (ICTL) and Regional Technology 
Outreach Programs are adequately funded and staffed to accomplish their goals and 
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objectives. 

Recommendations for the ICTL: 

• Continue to collect and analyze ITBS and TAP data in a longitudinal study. 

• Ask school districts in advance to setup and run area specific local studies in 
cooperation with institutions of higher education. 

• Standardize grant reporting formats for consistency. 

• Determine what performance indicators need to be measured over the nest three to 
five years to promote consistency in data collection and reporting to the legislature. 

Opportunities for the ICTL/Regional Technology Advisors: 

• Provide adequate professional-technical training for school district network system 
administrators. 

• Provide adequate training on integration of technology for school district teaching staff. 

• Provide consulting services to districts regarding technology planning, and training. 
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Appendix 1: Summary of the Data Collection Model 
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SUMMARY 



ICTL GOAL-INTEGRATION OF TECHNOLOGY 



Performance Outcomes: 

1 . Students will improve their academic performance as a result of the integration of technology. 

2. Students will improve their technological literacy as a result of exposure to technology. 

Performance Indicators: Data Source 

• Tier 1-ITBS/TAP test scores Riverside 

• Tier 2-Regional university research studies Regional Technology Advisors 

• Tier 2-Student exposure to technology Riverside 

• Tier 3-Qualitative descriptions of integration SDE-Technology Bureau 

Procedure: 

Tier 1 

1 . ICTL monies were made available in FY 1 995; therefore, we propose using the 1 994-95 school year 
as a starting point by which to track student progress. Student progress can be demonstrated over 
time by comparing the ITBS scores of all students who were in the 4th grade in the 1 994-95 school 
year and who will be taking the ITBS in the 1998-99 school year, and those who were in the 8th 
grade in the 1994-95 school year and will be taking the TAP in the 1998-99 school year. 

2. Identify students “technology concentrators” who are taking the ITBS or TAP test in 1998 as either 
being exposed to a “technology rich” background or not by using a short survey prior to the exam. 

3. Compare the academic gain scores among the groups showing academic improvement. 

4. Compare percentage of high, medium and low technology concentrators to show exposure. 

Note: More specific analysis could be accomplished in selected areas such as Math and English. 

Tier 2 

Identify Regional Studies that target and address specific academic performance outcomes. Aggregate 
and summarize results to show success of the performance outcome (integration of technology into the 
classroom). 

Tier 3 

Include qualitative data from grant proposals and technology plans which show integration of technology 
into the classroom curriculum. 

Note: The data from the grant proposals will go out in Phase II after State Board approval in June. The 
reports are due from the districts November 1 . 

■ ICTL GOAL-COMPATIBILITY 



Performance Outcome: 

Schools will be able to share information and resources through technology. 



Performance Indicators: 

• Tier 1 -Number of LANs and WANs 

• Tier 1 -Number of schools connected to the Internet 

• Tier 1 -Number of students using Internet 

• Tier 1 -Number of teachers using Internet 

• Tier 1 -Students participating in distance learning 

• Tier 2-Descriptive examples of compatibility 

Procedure: 



Data Source 

QED 96-97 Q#13A 
QED 96-97 Q#9 
QED 96-97/Phase II 

QED 96-97/Phase II 
SDVE/Phase II 

SDE-Technology Bureau 
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Tier 1 

Collect quantitative data on number of students using Internet, teachers using the Internet, students 
participating in distance learning, and student computers connected to a LAN. (Addition of a question on 
ITBS in October). 

Tier 2 

Glean descriptive information from ICTL grant proposals and reports that apply to the goal of compatibility. 

■ ICTL GOAL-TEACHER PREPARATION: 

Performance Outcome: 

Schools and Colleges of Education, working together, will effectively prepare instructors to teach using 
technology. 

Performance Indicators: Data Source 

. Tjer 1-Number of teachers taking assessments Regional Technology Advisors 

• Tier 1 -Performance on the assessments Regional Technology Advisors 

• Tier 1 -Extent to which in service training has been provided 

to teachers by school districts Phase II 

• Tier 1 -Extent to which in service training has been provided 

to teachers (contact hours/number served) Regional Technology Advisors 

• Tier 2-Descriptive examples of teacher preparation Regional Technology Advisors 

Procedure: 

Tier 1 

Collection of data would be done through the Regional Technical Advisors (RTAs). Each RTA would 
submit a report containing pertinent data. The data would then be summarized and used in the final report 
to the legislature. 

Data on district contact hours will be obtained via Phase II of ICTL Report. 

Tier 2 

Collection of the descriptive case would also be accomplished through the RTAs. Each RTA would submit 
a minimum of one study, with abstract, showcasing a relationship of successful teacher preparation 
practices or programs between the university and a school district. 

Note: A question needs to be added to Phase II of the grant/report, “How many training contact hours has 
your district provided teachers in the last year?” 

■ ICTL GOAL-COLLABORATION WITH COMMUNITIES AND BUSINESS 

Performance Outcome: 

Schools will involve community members, business, and post-secondary institutions in the implementation 
and use of technology in the schools. 



Performance Indicators: Data Source 

• Tier 1 -’’Other" dollars supporting technology SDE-Technology Bureau 

• Tier 1 -Money generated locally through technology bonds, District Survey 

grants, donations 

• Tier 2-Description of collaborative projects Regional Technology Advisors 
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Tier 1 

Quantitative data on private dollars will be collected through SDE. Moneys accrued through technology 
bonds, grants and donations could be collected on Phase II of the grant report. Data on dollars spent per 
student could be used as an additional indicator. 

Tier 2 

96-97 grant report summaries of successful programs already submitted to ICTL could be utilized or an 
additional question could be added to Phase II of the ICTL report process requesting examples. 

Note: Define “in-kind contribution” and develop a matrix for Phase II of the grant application ie: break out by 
federal, local bond and donated dollars. 

■ ICTL GOAL-TECHNOLOGY SYSTEMS ENHANCING EFFICIENT OPERATION OF THE 

SCHOOLS 

Performance outcome: 

Schools will use technology to improve administrative efficiency in schools operations. 

Performance Indicators: Data Source 

• Tier 1-Number of schools keeping track of students District Survey 

electronically (grades, lunch, discipline, attendance) 

• Tier 1 -Number of networked computers used for QED 96-97 Q# 1C 

administrative purposes. 

• Tier 1 -Number of schools submitting electronic reports SDE-Technology Bureau 

to SDE i.e. IBEDS, School Accreditation. 

Procedure: 

Tier 1 

QED data could be utilized to report the number of networked computers as well as number of computers 
used for administration as of 1996-97. 

Note: An addition to ICTL Phase II report could help acquire data on number of school districts who keep 
electronic student records. 

■ ICTL GOAL-TRAINING STUDENTS TO MAINTAIN TECHNOLOGY 

Performance outcome: 

Students will be able to install, maintain and support technology. 

Performance Indicators: 

• Tier 1 -Number of Technology Support Technician 
(TST) programs operating 

• Tier 1 -Student enrollment in TST programs 
(growth in enrollment) 

Procedure: 

Tier 1 

TST data can be utilized to show how the districts have been training students to help maintain, install and 
support technology. The number of programs as well as student enrollment in TST programs between 
1994 and 1998 would provide data on change in student involvement over time. SDVE currently has the 
data needed. A summary will be provided. 



Data Source 

State Division of 

Vocational Education (SDVE) 

SDVE 
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Appendix 2: Data Collection Matrix 
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Appendix 3: IFARMS Technology Expenditure Master 




42 245 



IFARMS TECHNOLOGY EXPENDITURE ADDENDUM MASTER 
FORM B 
FY 1097-98 
FUND 246 



IFARMS Categwy/ICTL Goal 
100 Salaries 



Goal 1 - imitation 

Goal 2 * Compatibility 

Goal 3 • Collaboration w/ Colleges 

Goal 4 - Collaboration w / Community 

Goal 5 - Tachnotogy Syttsme 

Goal 6 • EvaHietion 

Goal 7 * Student Training 



Totals 



irrms w 

$94,644.26 

$92,140.61 

$30,307.47 

$224,633.61 

$36,234.33 

$106,864.19 



Goal o - system* auppon 

IToui ioo salary 1 


Tff«5iW5381 


200 Benefits 


Goal 1 - Integration 

Goal 2 -Compatibility 

Goal 3 - Collaboration w/ College* 

Goal 4 - Collaboration w/ Community 

Goal 5 - Tach oology Syttama 

Goal 6 - Evaluation 

Goal 7 * Student Training 

Goal 6 - Syttama Support 


$26,614.47 

$22,359.86 

$20,274.08 

$5,896.50 

$36,838.60 

$7,762.23 

$23,376.72 

$70,805.47 


[Total 200 Gifltfti i 


138552735] 
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Goal 2 - Compatibility 

Goal 3 - Collaboration w/ Collagat 

Goal 4 - Collaboration wf Community 

Goal 5 - Tachnotogy Syttama 

Goal 6 - Evaluation 

Goal 7 - Studant Training 

Goal 8 - Syttama Support 


$229,970.10 

$161,745.44 

$149,333.83 

$29,454.69 

$330,709.25 

$17,195.55 

$19,231.78 

$289,655.24 
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Goal 1 - Integration 

Goal 2 - Compatibility 

Goal 3 - Collaboration w/ Collagaa 

Goal 4 - Collaboration w/ Community 

Goal S - Technology Systeme 

Goal 6 - Evaluation 

Goal 7 - Student Training 

Goal 8 - Syttama Support 


$387,003.32 

$164,725.64 

$21,128.68 

$24,613.93 

$550,320.95 

$21,563.95 

$62,651.23 

$226,979.74 
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500 Capital Otolaeta ... 


Goal 1 • Integration 

Goal 2 - Compatibility 

Goal 3 - Collaboration w/ Collegea 

Goal 4 - Collaboration w/ Community 

Goal 5 - Technology Syaleme 

Goal 8 - Evaluation 

Goal 7 -Student Training 

Goat 8 - Syateme Support 


$2,623,501.67 
$954,455.70 
$60,387.21 
$121,620.45 
$1,599,275.48 
$61,537.88 
$126,602.49 
. $204,202.72 
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ICTL DoUam Spent on Each Goal 



1 


— 13,640, 7flgre 


ssrar- 


2 


$1,306,130.00 


13.02% 


3 


$343,264.41 


3.42% | 


4 


$212,003.04 


2.11% 


5 


$2,741,777.80 


27.31% 


6 


$144,203.04 


1.44%, 


7 


$338,046.41 


3.38%i 


8 


$1,221,473.08 


12.17%' 


* 


— $10,040,775.15 


1 Tod.06%; 



• Figure# Reflect 106 of 112 Districts Reporting 
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Appendix 4: Technology Exposure Questionnaire 
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Technology Exposure Survey 

Directions: Listed below are questions which are designed to measure the amount of exposure you have 
had to technology over the past four years. Please read each question carefully, then mark the response 
box that best describes your experience on the test answer sheet in the student ID number box. Then write 
the answer you chose in the empty box below the number of the question in the student ID number box. 
This is NOT a test so there are no right or wrong answers; all answers will be kept confidential. 

Example: 

How much opportunity have you had over the past four years to write stories using the computer? 

0 = A lot 1 = Some 2 = Not Very Much 3 = Almost None 

The student in this example felt that she had a lot of opportunity over the past four years to write stories on 
the computer for class assignments, so she filled in the response "0" on her test answer sheet. Then she 
wrote the answer "0" in the empty box below the question in the student ID number box. 



1. How capable are you at choosing the appropriate tool for completing a project; for example, using 
a word processor for a report or software like PowerPoint or HyperStudio for a class or group 
presentation? 

0 = Very Capable 1= Somewhat Capable 2 = Not Very Capable 3 = Poor 

2. Over the past four years, the teachers in my school have discussed legal, ethical, and acceptable 
uses of computers in our classroom. 

0 = Alot • 1=Some 2 = Not Very Much 3 = Almost None 

3. How much opportunity have you had over the past four years to design, develop, or present class 
projects using technology (i.e. web pages, video, presentation software, etc.)? 

0 = Alot 1= Some 2 = Not Very Much 3 = Almost None 

4. Over the past four years, how much have your teachers discussed with you the advantages and 
disadvantages of relying on technology for completing classroom assignments? 

0 = A lot 1 = Some 2 = Not Very Much 3 = Almost None 

5. How capable are you using computer software to simulate real-world situations such as taking a 
historic journey back in time, building a city, or investing in the stock market? 

0 = Very Capable 1 = Somewhat Capable 2 = Not Very Capable 3 = Poor 

6. How much opportunity have you had over the past four years to work with other students using 
technology to contribute to a project? 

0 = Alot 1= Some 2 = Not Very Much 3 = Almost None 

7. How much opportunity have you had over the past four years to identify and solve routine problems 
(troubleshooting) related to making a computer work? 

0 = A lot 1=Some 2 = Not Very Much 3 = Almost None 

Over Please © 

8. How capable are you at using e-mail and Internet to help complete classroom assignments and 
projects? 

ERIC 
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0 = Very Capable 1 = Somewhat Capable 2 = Not Very Capable 3 = Poor 

9. Over the past four years, on average, I use the computer at home per week? 

0 = 8 or more hours 1 = 4 to 7 hours 2 = 1 to 3 hours 3=0 hours 

10. Over the past four years, on average, I use the computers at school per week? 

0 = 8 or more hours 1 = 4 to 7 hours 2 = 1 to 3 hours 3=0 hours 
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Appendix 5: Additional Example Summaries of Higher Education 

Research 
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“Completed" Studies/Projects 



Boise State University 



Effects of Technology and Curriculum Integration on 3 rd Grade Students’ Technology, 
Thinking, and Science Process Skills 

Third grade students, of mixed ability, were used to evaluate technology skill 
development. Pre and posttests were conducted prior to and immediately after instruction. • 
Following their project research, students developed database, spreadsheet, and word 
processing documents. Collective data was extensive in all areas of the study. Conclusions 
revealed that all students, through the use of technology, showed measurable improvement in 
self-efficacy and objective knowledge. In addition, all classes showed significant growth in 
applied descriptive vocabulary. 



A Comparative Study of Geometry Achievement using Geometer’s Sketchpad 

The study attempted to demonstrate that students using specific software gained a 
greater understanding of geometric principles than those students who do not use the 
program. Two sections of a geometry class used Geometer's Sketchpad while two sections 
did not use the program. The same teacher taught all sections and the achievement of both 
sections was compared. While a significant difference was noted between groups, it was 
concluded that there was no appreciable interaction among the achievement levels. This 
indicated that levels remained constant between the instruction methodology. 



Vallivue High School PLATO Pilot Research Project 

This study, using at-risk students, attempted to demonstrate increased achievement in 
content area courses using the PLATO system compared to progress in traditional classes. 

Pre and posttest data was collected over a year’s time and measured for gain in three 
categories (Reading for Understanding, Sentences, and Envelope Writing). Sixty-four percent 
of the students participating in the study showed measurable gains in all three areas. 



Does the Use of Hypermedia Software Affect the Learning of 6 th Grade Social Studies 
Students? 

This study was designed to investigate the effectiveness of computers in improving test 
scores, time-on-task efficiency, and content retention. Data collection involved forty-six 
students, divided into a control group and experimental group. Two instructional units over a 
period of five weeks. Students in the experimental group used “HyperStudio" to access 
information and to create a report. Pre and posttest results indicated a marked improvement in 
daily worksheet grades for the experimental group. However, no significant difference was 
measured on chapter and unit pre and posttest scores. 

O 47 
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University of Idaho (Moscow) 

Investigating the Use of the Internet as an Alternative Instructional Delivery Mode 

A total of fifty-one students, in three groups, were used in the study. Pre and posttest 
data was collected from all groups that received the same amount and quality of instruction for 
all modules. Statistical analysis indicated no significance between the groups. However, this 
is considered to be a positive result as it suggests that the Internet group performed as well as 
the other groups. The inference being that, carefully developed and implemented, the internet 
is a viable, alternative instructional delivery mode. 



Investigating the Use of Microcomputers as Enhancement Aids to Facilitate Lessons by 
Teachers 

Experimental and control groups involving fifty-three elementary students participated 
in the study. Similar instruction was delivered to both groups. Following the instruction, 
depending upon the groups, students were then allowed to use either the computer with 
appropriate software or traditional worksheets and workbook activities as enhancement aids. 
The Iowa Test of Basic Skills (ITBS) was used to gather pre and posttest data. Statistical 
analysis favored computers as effective enhancement aids suggesting that computers assist 
student learning as related to the content areas of mathematics and language arts. 



“In Progress” Studies/Projects 



Boise State University 

Vallivue High School PLATO Research Project 

Using a class of special needs and traditional students, the PLATO system will be 
utilized to assess the effectiveness of a computer program in meeting the needs of students 
assigned to or requesting the Resource Room. The study will focus on whether at-risk and 
traditional students using the PLATO system and appropriate software demonstrate increased 
achievement in content areas as compared with traditional teaching methodology. 

Patterns of Technology Access 

The aim of this study is to identify distribution patterns of technology for special needs 
students compared to normally achieving students, and examine how these patterns and 
arrangements may affect student outcomes and accessibility for all students. This takes into 
account such students with disabilities and those with limited English speaking proficiency. 
Collected data will include specially prepared surveys from students, teachers, and district 
personnel. 



c 



48 



O 



252 



1 ) 0 ( 101)0001599 



Idaho Technology Initiative - Accountability Report 

Technology Based Writing Partnerships Between 6 th and 12 th Grades and 8 th and 11 th 

Grades 

Telecommunications (e-mail) is the basis for this study. Students using e-mail will be 
compared with students not using e-mail to determine if writing skills will show a measurable 
difference over a given period of time. Writing partnerships will be formed between 6 th and 
12 th grade students and 8 th and 11 th grade students. In addition to pre and posttests for 
assessments, each student will be required to complete an electronic writing portfolio. Data 
comparing previous non-technology based communications will be evaluated along with an 
assessment of changes in writing grade levels. 



Database & Science: Integrating Databases into the 5 th Grade Science Curriculum 

This study employs ten 5 th grade teachers in the Meridian School District. The 
experimental group consists of five classrooms with access to computers. The control group 
will not use computers. The content and objectives of the unit are based upon the current 
science textbook and the focus of the study is to measure differences in science learning. Pre 
and posttest assessment will be chapter and unit tests from the assigned science textbook. 



Do Students Who Have Access to Word Processors Learn Compositional Writing Skills 
Better Than Students Who Use Manual Writing Process Techniques? 

Two groups (A&B) will be used to investigate student achievement and efficiency in 
compositional writing. Experimental Group A will use word processing software to complete all 
phases of a student writing assignment. Control Group B will carry out the same assignment 
using paper and writing instruments. Note taking, first drafts, and final papers, along with the 
8 th grade writing competency exam scores will be used as dependent measures for 
comparison. 



Do 11 th and 12 th Grade Students Doing Research, Note Taking, and Technical Writing 
Experience Better Achievement Using AlphaSmart Portable Word Processors than 
Manual Techniques? 

The purpose of this study is to Investigate the uses of technical writing aides in 
comparison to manual techniques in helping students achieve at higher levels. Writing classes 
will be divided into two groups (A&B). The experimental group (Group A) will use AlphaSmart 
portable word processors while Group B will complete the same assignment using paper and 
writing instruments. All materials will be collected and analyzed along with time-on-task 
observations for dependent measures in comparison. 

Do Sixth Graders' Show Greater Technology Competency Acquisition Gains when 
Computers are placed in a Classroom or in a Laboratory? 

Technology instruction either: a) integrated into the classroom curriculum with 
computers in the classroom; or, b) throughout direct instruction of technology curriculum in a 
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computer lab will be used to determine the impact on 6 th graders’ computer competency. 
Criterion-referenced technology competency exam scores will compare student technology 
achievement. Student satisfaction surveys and observations of time-on-task will also assess 
productivity and efficiency. 

University of Idaho (Moscow) 

Investigating the Effect of Hypermedia on 4 th Grade Student Achievement in 
Mathematics 

A design is being implemented to use a group of students who were taught without the 
benefit of Hypermedia software. A new group will be taught the same content with the addition 
of specific Hypermedia software to enhance instruction. Following instruction, students will be 
assessed using pre and posttests to determine if there is a significance difference between 
students using Hypermedia to enhance mathematics instruction and students who do not. 



To What Extent Does the Technology Enhanced Classroom Impact Student 
Achievement in Language Arts? 

An experimental design using traditional control/experimental groups will be used to 
compare student achievement. Groups will receive different instruction and/or have access to 
specific technology and software. The groups will be evaluated using the State Direct Writing 
Assessment to determine achievement. 



Idaho State University 

Does teacher training, in a specific mathematical problem solving software positively 
impact the acquisition of mathematical problem-solving skills at the upper elementary 
level? 

Teachers were trained during day-long, hands-on workshops with the specific software. 
They were required to design a lesson incorporating the software into their mathematics 
curriculum, teach the lesson, and then report the results. Thus far, reported results are 
anecdotal and positive. Pretesting is currently underway to collect quantitative results from 26 
classrooms using the program. 

Will training in a specific geography software program positively impact the acquisition 
of geography-related concepts in the secondary classroom? 

Teachers were trained during day-long, hands-on workshops with the specific software. 
Teachers were required to design a lesson incorporating the software into an appropriate area 
of their curriculum, teach the lesson, and then report the results. Teachers anecdotal reports^ 
show that student interest ranged from "mildly interested” to "actively interested and engaged”. 
Pretesting is underway to collect quantitative data from 35 secondary teachers using the 
program. 
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What changes occur in teacher knowledge, attitude, and implementation of technology 
as a result of specific multi-media training? 

Through the Just-in-Time Technology Challenge Grant, teachers were given specific 
training in incorporating multi-media software into classroom instruction. Approximately 240 
teachers were selected to take part in the first year of the project, and were involved in a four- 
day training session. Pre-test information has been collected on teacher attitudes, technology 
use, and self-assessment. Student training and project development begins in January of 
1999. 
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Appendix 6: Additional Examples of School District Research 
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SCHOOL DISTRICT PROJECT SUMMARIES 



Boise School District #1 

■ Using the Waterford Early Reading Program, two schools and 1 55 students were involved in a study 
designed to evaluate the impact of specific software on the learning process. A control group and 
experimental group were used and pre/posttests were administered. The results indicated that the 
experimental group (using Waterford Early Reading Program) outperformed the control group by a 
significant margin. 



Glenn’s Ferry School District #192 

■ Two classes of advanced anatomy students were used in a study to see if appreciable gains would 
occur in academic performance with one class using laptop computers and the other class being 
taught in a traditional manner. In addition to increased academic performance demonstrated by the 
class using computers, it was also noted that there were improvements in time-on-task, motivation, 
and attendance. Students using computers were also required to make a multi-media presentation 
as part of the class requirement. 



Valley School District #262 

■ Computers were used with mentally challenged students working on multiplication tables. 

Previously, these students were unmotivated by traditional techniques and displayed little progress 
over a period of one and a half years. Given computers and appropriate software there were noted 
differences in motivation. Students appeared to enjoy what they were doing and demonstrated 
increased motivation and positive attitudes. Learning increased significantly as evidenced by 
average pretest scores of 72% compared to average posttest scores of 89%. 



Emmett School District #221 

■ This study focused on a self-confidence survey administered to all teachers. Based on the individual 
results, each teacher was able to diagnose and prescribe a professional development plan for 
technology. Classes, seminars, and work sessions dealing with computer skills and integration were 
generated. The emphasis on training and individual study resulted in improved confidence and 
increased computer literacy. 



Clark County School District #161 

■ A district-wide assessment was conducted to determine student (K-12) learning preferences with 
computers as a given choice. Teachers were also surveyed to determine to what degree and level of 
usage computers were integrated in the classroom. Both students and teachers indicated positive 
attitudes in the use of technology and a majority of students chose computers over other forms of 
learning. Using classroom observations, teachers determined that students strengthened, improved, 
and reinforced skills through the use of the computers. 



Firth School District #59 
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■ A study was conducted to determine if certificated district personnel after using computer technology 
and software has shown significant improved technology competency. Measurement was conducted 
through the use of a pre/posttest on computer literacy and skills. Six classes in tech training, access 
to internet, telecommunications (e-mail) were offered district staff and teachers. Following the 
training, knowledge and skill levels increased by 30%. 



Blackfoot School District #55 

■ This study involved twenty-one principals from rural districts. The purpose of the project was to show 
how attitudes and personality types influenced implementation of technology by school 
administration. Principals received training in the use and integration of computers. A pre/posttest 
was administered. It was determined that there was no significant correlation between attitudes and 
training. However, principals did maintain a positive attitude toward the implementation of 
technology. Also expressed was the opinion that technology training for teachers is inadequate. 



Hansen School District #415 

• Using specific software (SkillsBank, Accelerated Reader), grade level trends were monitored using 
ITBS and TAP scores to determine gains in the areas of math, language arts, and reading. While 
the study is still in progress, initial results indicate that grade levels using SkillsBank (math, language 
arts) showed the most gains in their ITBS scores. Because Accelerated Reader was recently 
adopted, it has not been evaluated for significant improvements in standardized test scores. 



“IN PROGRESS SUMMARY” 
SCHOOL DISTRICT REPORTS 



Several school districts submitted reports of studies that were still in progress. They reflected a 
range of creativity and resourcefulness in measuring the benefit that the integration of computer 
technology and software has on student performance in the classroom. 

The extent of these on going studies includes a cross section of elementary, middle school, and 
secondary grade levels. The expanse of curriculum reported under study involves language arts, 
reading, science, business, and technical writing. A focus of in-progress studies also includes 
comparative data on basic skills as they relate to ITBS test scores tracked over a four-year period, 
evaluation of technology competency acquisition, a correlation of technology integration with grade 
performance, attendance, and discipline, and teacher attitudes. 
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Glossary 



Core Battery - a subset of 7 to 9 tests which are included in the Complete Battery. For level 
5-8 the Core includes tests on vocabulary, word analysis, reading, listening, language, math 
concepts, and math problems. For the level 9-14 the Core includes tests on vocabulary, 
spelling, capitalization, punctuation, usage and expression, math concepts and, estimation, 
math problems, and math computation. 

Effect Size - the degree to which the effect is present in the population. I.e how many 
standard deviations separate the effect means. 

Standard Score - the Developmental Standard Score is a number which describes a student’s 
location on an achievement continuum. The main advantage to using the Standard Score 
instead of the Grade Equivalent Score is that it better mirrors reality. I.e. the Grade Equivalent 
Score shows an equal average growth between any pair of grades. 

Gain Score - the difference between the Standard Scores of two data points. 
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West Virginia Study 

Under what conditions Is technology effective in advancing 
learning and teaching? j , L . tu 

This is the fundamental question that drives the research of the Milken 
Exchange on Education Technology. 

Education technology evolves quickly because of changes in 
technology and advances in our understanding of how to use it, 
engage, challenge and nurture learners. The Jag between the 
introduction of a program and evidence of its effectiveness may be 
significant. Still, It is useful to know what kinds of education 
technology work and in what ways. 

This study of West Virginia's Basic Skills/Computer Education is a 
collaborative investigation by the West Virginia 
Education , the educators and students in the schools studied, the 
Milken Family Foundation, and Interactive Inc. 

The findings suggest that Virginia's Basic Skills/Computer Education 
program has had a positive impact on learning. West Virginia has had 
across-the-board increases in statewide assessment scores in all basic 
skills areas, and their NAEP (National Assessment of Educational 
Progress) scores have gone Up. (Please download a copy of the report to see where 
gains were noted and how technology was Integrated Into the curriculum.) 

The West Virginia Story's authors: 



Dr. Dale Mann, a professor at Teachers College, Columbia University 
and founding chair of the International Congress for School 
Effectiveness, an organization with members from more than half the 
countries of the world that is dedicated to improving schooling for the 
neediest children; 

Dr. Charol Shakeshaft, a professor at Hofstra University. She is a 
specialist in research methods and a pioneer in the field of gender 
equity and schooling. Dr. Shakeshaft is the author of a leading 
textbook on women in educational leadership. 

Jonathan Backer. J.D. is a research specialist in law and education. 
A doctoral student at Teachers College, Columbia University, he is 
interested in social science research utilization in the educational 
policy context. 

Dr. Robert Kottkamp is a professor in the Department of 

261 

http^/www.mff.org^udt*cWprojecl#.taf?Junction=d«tall&Conl«nLuld1«161 (1 of 2) (4/24/2001 



2:39:48 PM] 

BEST COPY AVAILABLE 




! 

West Virginia Study 




Administration and Policy Studies at Hofstra University where he 
teaches research methods, administrative leadership and reflective 
practice. 



Publications West Virginia Story 

3/10/99 by Dale Mann, Ph.D., Charol Shakeshaft, Ph.D., 
Jonathan Becker, J.D., Robert Kottkamp, Ph.D. 

This study examines West Virginia's long-running Basic Skills/Computer 
Education program and its positive Impact on students' standardized test 
scores. 



Articles West Virginia Study Reaulta 

3/10/99 Exchange study finds direct link between use of learning technology and 
higher academic achievement In West Virginia. Students raised their 
standardized test scores In math, reading and language arts since the state 
Implemented Its Basic Skills/Computer Education Program. 
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Under what conditions is technology effective in advancing learning and teaching is the fundamental question that 
drives the research interests of the Milken Exchange on Education Technology. The West Virginia Basic Skills/Computer 
Education (BS/CE) technology implementation beginning nearly ten years ago deserves our scrutiny because of its 
scale, consistency and focus. What can we learn from an approach that spanned a whole state for a full decade? The 
issues of system design, training, technology capacity, technical support, and means of measurement are all 
powerfully present in the West Virginia experience. The goal of the Exchange in commissioning the study was not to 
praise it or criticize it, but to understand it and to make that understanding known to others. 

The findings of this study suggest strongly that the BS/CE program had a powerfully positive effect in West 
Virginia, especially in those schools that used it most intensively. 

• Significant gains in reading, writing, and math were achieved. 

• BS/CE was found to be more cost-effective than other popular interventions including class-size reduction. 

• The program was especially successful with low income and rural students as well as with girls. 

But these findings need to be interpreted cautiously by educators and policymakers because: 

• BS/CE is based on instructional learning systems designed over a decade ago and limited to the then 
available technology. For example, easy access to the Internet was just a dream in 1989. 

• The pedagogy upon which instructional learning systems are based makes little use of project-based 
learning and other constructivist curricular approaches that are the leading edge of learning technology today. 

• BS/CE fit the learning and teaching realities of West Virginia over the last decade. That does not make it 
appropriate for every district or state where the characteristics of learners and teachers may be quite different. 



Learning technology evolves quickly because of changes in technology and advances in our understanding of how to 
use it to engage, challenge and nurture learners. The lag between the introduction of a program and evidence of its 
effectiveness may be significant. Still, it is useful to know what works and in what ways. The future forms of 
learning technology are impossible to predict, but we can design ihem better based on the islands of research that 
help explain where we have been. 



Cheryl Lemkc 
Executive Director 

Milken Exchange on Education Technology 
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The West Virginia Department of Education is pleased to collaborate with the Milken Family Foundation in the 
publication of a landmark study documenting the powerful and positive impact of the Basic Skills/Computer 
Education program on student achievement in West Virginia. 

Our experience and data tell us that the Basic Skills/Computer Education program is successful. West Virginia has had 
across-the-board increases in statewide assessment scores in all basic skills areas, and our NAEP scores have improved. 
Just as importantly, we have seen the faces of children light with excitement and learning and have heard the 
renewed enthusiasm of teachers when technology is integrated into the curriculum. 

The reasons for West Virginia’s success are numerous. Clearly articulated goals focus upon increased student achieve- 
ment in reading, mathematics, and composition. Software .was aligned with the West Virginia Instructional Goals and 
Objectives. Implementation, which began in the earliest grades, moves upward each year. Comprehensive and 
timely staff development enables teachers to correlate the software with the curriculum and integrate the technology 
into instruction. The state-purchased computer systems are distributed equitably and provide a consistent 
platform for instructional and support purposes. Administrative support at the county, region, and state board of 
education levels provides clear evidence of program importance. All of these factors were developed through partic- 
ipating stakeholders who provided a bottom-up approach that created a statewide solution for technology integration. 

West Virginia's model calls for increasingly higher standards for student achievement. As our model continues to be 
buttressed by implementation of best practices identified in solid research, West Virginia students will progress to 
meet these higher standards. This study adds to the limited body of solid research on the impact of learning tech- 
nologies on student achievement. 

The Milken Family Foundation is to be commended for promoting research aimed at identifying those factors that 
contribute to student achievement with the use of learning technologies. This study is a collaboration among the 
West Virginia Department of Education, the educators and students in the schools studied, the Milken Family 
Foundation, and Interactive, Inc. This collaborative effort is truly appreciated. Increasing student achievement remains 
our central focus, and the effective implementation of technology will continue to advance this goal. 

Henry R. Marockie 

State Superintendent of Schools 

West Virginia 
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WEST VIRGINIA STORY: 
Achievement: gains from 
a statewide comprehensive 
instructional technology program 



EXECUTIVE SUMMARY 



The Program. 

West Virginia's "Basic Skills/Computer Education" program is unique in its eight year longevity and in its 
documented student achievement outcomes. This report describes the program and connects its features to 
gains in student test scores that are practically and statistically significant. 

The Basic Skills/Computer Education (BS/CE) program was authorized in 1989-90 and, beginning with the 
kindergarten class of 1990-91, hardware and software were installed in schools and teacher training began. 
The program consists of three basic components: 

(1) software that focuses on the State's basic skills goals in reading, language arts and mathematics; 

(2) enough computers in the schools so that all students will be able to have easy and regular access 
to the basic skills software; and 

(3) professional development, for teachers in the use of the software and the use of computers in general. 

Each year from 1990-91 onward and beginning with kindergarten, at a cost of about $7 million per year, the 
State of West Virginia provided every elementary school with enough equipment so that each classroom 
serving the grade cohort of children targeted that year might have three or four computers, a printer and a 
school-wide, networked file server. Schools could choose to deploy the computers in labs and centers or dis- 
tribute them directly to classrooms. As the 1990-91 kindergarten class went up the grades, so did the 
successive waves of new computer installations coupled with intensive professional development for 
teachers and software chosen from either IBM or Jostens Learning. 

Research Methods. 

We collected data from all fifth graders (950) in 18 elementary schools that were selected to represent the 
range of variables that might influence technology use and student achievement, e.g., intensivity of BS/CE 
use, software vendor, student prior achievement and sociodemography. The 1996-97 fifth graders had the 
most complete test score records and they were the first cohort to have had the consistent availability of 
BS/CE across their entire school experience. The sample size supports generalization at the 95% level of 
statistical confidence. We collected survey data from 290 teachers in the study schools. Data were both 
quantitative (state and publisher's test files, survey results) and qualitative (on-site field documentation, case 
analysis, interview results). 




11 

263 



000000001715 



Student test data were scaled scores on the Stanford-9 achievement test. Because scaled scores are 
normed against a nationally representative group, they are appropriate for comparison purposes and for the 
computation of gain scores. Our emphasis is on gains in "basic skills," a score computed and reported by 
the West Virginia Department of Education for comparison purposes, and a score that measures the math, 
reading and language arts areas that were the focus of BS/CE. 

Explaining Test Scare Variation: The Access/Attitude/Training Model. 

We used factor analysis to search for input phenomena that were grouped both conceptually and in terms of 
respondent perceptions and that were also related to variation in student test scores. The three components 
of this empirically-derived model — access, attitude, and training — are similar to what leaders in instructional 
technology advocate. Unless there is sufficient equipment and opportunity to learn (access) there are unlike- 
ly to be effects from instructional technology. And, unless teachers have a chance to learn about the 
technology and how it may make them and their students more successful (training), they are less likely to 
believe it can help (attitude) and, in fact, to use it. 

BS/CE Effect on Student Achievement. 

The more of each of the model components that the student experienced, the higher the gain score on the 
Stanford-9. The BS/CE technology regression model accounts for 1 1% of the total variance in the basic skills 
achievement gain scores of the 5th-grade students. That impact is powerful and practically significant; it is 
also statistically significant at more than the .001 level. More to the point, an 1 1 % improvement in test scores 
would be welcomed by most students, parents and educators. 

But, we believe that the 1 1% explained by our model underestimates the value added by instructional tech- 
nology. First, schools are only one of several educators. Family background explains a great deal about why 
some children do better than others. Since James Coleman's 1965 research, we have known that childrens 
achievement in school is conditioned more by what they bring to school than by what schools are able to do 
with them. Consider: 

(D)espite the wide range of diversity of school facilities, curriculum and teachers, and despite the wide 
diversity among student bodies in different schools, over 70% of the variation in achievement for each 
group is variation within the same student body (James S. Coleman, et al, Equality of Educational 
Opportunity. Washington DC., US Department of Health, Education and Welfare, 1966, p. 77). 

If, as Coleman and others assert, 70% of the variation in test score performance relates to family and other 
home and background factors, that leaves 30% that schools can influence. We call that 30%, "school acces- 
sible performance." 

This analysis documents that as much as 11% of the gain score variance for one year can be explained by 
BS/CE. Thus, BS/CE explains more than a third of the "school" reasons why students' achievement scores 
improved. Said another way, of all the factors that can affect a child s learning, about 30% is within the 
school's sphere of influence. The BS/CE technology initiative explained nearly 1 1% of that 30%. 
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There are two additional reasons to believe that the reported gain scores underestimate the total effect of 
BS/CE. The 11% represents gain score variation explained by BS/CE within a single year, i.e., 1997 to 1998. 
However, these fifth graders had BS/CE for four previous years. Thus, the cumulative effect of BS/CE most 
likely accounts for more than the 1 1 % of the single year gains that we found. 

We also suspect that the gain scores reported are unnecessarily conservative because current hardware and 
software is more powerful than what was implemented in BS/CE's early years. Current technology is likely 
to yield even larger gains. 



The Distribution of Achievement Gains Across Groups of Children. 

BS/CE helped all children perform better, but the data indicate that BS/CE helped the neediest children the 
most. Those children without computers at home made the biggest gains in (1) total basic skills, (2) total 
language, (3) language expression, (4) total reading, (5) reading comprehension, and (6) vocabulary. Also, 
the Access/ Attitude/Training model explains more of the basic skills gain scores for students who report 
lower grades. 

Although the relative disadvantage of girls is a regularity of the technology and the gender literatures, girls 
and boys reported the same access and the same use of computers in West Virginia. The more years that 
girls report having used computers, the more they like them and the more they report knowing about them. 
(Students reported increased computer use every year from kindergarten through the fourth grade.) Girls 
reported that computers were more accessible to their particular learning needs than were their teachers. In 
math and reading outcomes, there were no gender differences. 



Teachers and B5/CE. 

More than half the State's teachers are confident in using computers in their teaching, only 1 9% are not. Half 
the teachers thought that technology had helped "a lot with West Virginia instructional goals and objectives." 
Almost half the teachers became more enthusiastic about BS/CE as time passed. 



Implementing BS/CE. 

BS/CE was fielded as it was designed— with a critical mass of hardware, software and training tightly focused 
on a grade-by-grade follow through schedule and on basic skills acquisition'. 

Schools could choose how to deploy the BS/CE computers— concentrated in labs, distributed to classrooms 
or a combination of lab-plus-classroom distributed. The deployment choice reflects compliance with the cur- 
rent goal of "technology integration into classrooms," and is thus significant for policy and crucial to teachers. 

Students who had access to BS/CE computers in their classrooms (the "distributed" pattern) did signifi- 
cantly better than students who were taught with BS/CE equipment in lab settings. They had higher gains 
in overall scores and in math. They also scored higher on the 1998 tests than did those in labs. (There were 
no differences on scores by source of software.) 
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Teachers who had computers in the classroom reported higher skill levels in delivering instruction, 
planning lessons, managing paperwork and word processing. Teachers who had BS/CE computers in 
their classrooms also reported more time using BS/CE computers for reading, math and writing instruction 
than either of the other two patterns. Sixty-one percent of the teachers with access to computers in their 
own classrooms said they were confident in using computers in their teaching compared to only 43 percent 
of the teachers who took their children to a lab for instruction in, about, or with computers. 

Additional Explanations for Test Score Gains. 

During the period studied, the State renovated 470 schools and built 68 new buildings. Between 1990 and 
1 993, increases to the average teacher's salary moved West Virginia up from 49th to 34th among the states. 
Still 48% of the teachers chose "technology" as the number one explanation for student learning gains. 

The state also required or instituted "Unified School Improvement Planning," a statewide curriculum frame- 
work ("West Virginia Instructional Goals and Objectives"), local school councils, faculty senates, school site 
accreditation visits and a “probationary" procedure. Also during this period, West Virginia Bell connected 700 
of the State's 840 school buildings to Bell Atlantic's "World School" Internet service. The measured pres- 
ence of "other technology related initiatives" accounts for 0.4% of the variance in achievement scores. Each 
of those initiatives probably accounts for parts of the total improvement although it was beyond the scope of 
this analysis to establish those amounts. 

Conclusions. 

This analysis establishes how much value can be added on a statewide basis from a sustained instructional 
technology initiative. The data indicate that as much as a third of the gains in "school accessible achievement" 
can be powered by instructional technology. The data also signal aspects of policy strategy and tactics. 

BS/CE is scalable. The expenditure proportions are not beyond reach for other states. (See also Lewis Solmon's 
"Afterword.") The outcomes are established. The program's components have been well documented. 

As other jurisdictions consider instructional technology as an agent of improvement, is it of interest that a 
package of hardware/software/process innovations can account for a large fraction of the test score improve- 
ment that is available to public policy intervention? And is it of further interest that, in addition to test score 
gains, those innovations can help position children for a technologically demanding economy, society and polity? 

BS/CE also has a number of features that are uncommon in the state education policy landscape. The 
features that deliver an installed base critical mass depart from the norm of a small number of computers 
equitably distributed across a large number of classrooms. The choice of software from a fixed set of two 
vendors departs from the conventional ceding of choice among hundreds of vendors to hundreds of schools 
(and often, to thousands of teachers). We believe that part of the explanation for BS/CE's success is the 
defined focus of its implementation. 

Still, policy choices, political choices always honor local values. In American schooling, thousands of juris- 
dictions make their own choices. It may be that this documentation of the student outcomes associated with 
West Virginia's program of Basic Skills/Computer Education will advance the consideration of similar initia- 
tives in other jurisdictions. 
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WEST VIRGINIA 5TORY: 
Achievement gains from 
a statewide comprehensive 
instructional technology program 



1.U INTRODUCTION 

West Virginia's instructional technology initiative is unique in its eight year longevity and in its documented 
student achievement outcomes. This report describes the results of quantitative and qualitative analyses con- 
ducted with the support of the Milken Exchange on Education Technology across a stratified sample of 
schools in the state. 

1.1 West Virginia's "Basic Skills/Computer Education" (BS/CE) Program 

The Basic Skills/Computer Education (BS/CE) program was authorized in 1989-90 and, beginning with the 
kindergarten class of 1990-91, hardware and software were installed in schools and teacher training began. 
BS/CE was intended to improve the basic skills learning of West Virginia's elementary students through tech- 
nology. The program consists of three basic components: (1) software that focuses on the States basic skills 
goals; (2) enough computers in the schools so that all students would have easy and regular access to the basic 
skills software; and (3) training for teachers in the use of the software and the use of computers in general. 

Each year from 1990-91 onward and beginning with kindergarten, the State of West Virginia provided every 
school with enough equipment so that each classroom serving the grade cohort of children targeted that year 
might have three or four computers, a printer and a school-wide, networked file server. Schools could choose 
to deploy the computers in labs and centers or distribute them directly to classroom^. As that kindergarten 
class went up the grades, so did the successive waves of new computer installations coupled with intensive 
professional development for teachers and software chosen from either IBM or Jostens Learning. The soft- 
ware and training emphasized the basic skills of reading, mathematics and writing. 

The BS/CE that students and teachers experienced was an obvious amount of new gear (either concentrat- 
ed in a showcase lab or center or distributed directly to classrooms); new software that related directly to a 
consistent, statewide priority on basic skills instruction 1 ; and intensive training prior to implementation cou- 
pled with continuous support during the early implementation. Ail elementary teachers on a given grade level 
were experiencing the same software, the same expectations, the same new challenges, and the same 
opportunities. BS/CE thus covered all the bases of technology integration— hardware, software, and pro- 
fessional development and involvement— and all in a concentrated, sustained and visible program. 

1 This analysis wa? designed asT study oi the aggregate impact oi all of BS/CE’s features on educational achievement. It was not designed to measure the rela- 
tive merits of the two software vendors, IBM and Jostens. The software packages that each made available were targeted on basic skills instruction. To make the 
software useful for teachers and to relate it to state priorities, the vendors mapped their packages onto the State’s curriculum objectives. Finally, each package 
included Integrated Learning Systems features— student and class data reports to support individual, classroom, grade-level, and school-wide analysis and action. 
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BS/CE is important because of how it was done — the inputs — and because of what happened — the 
outcomes for students, teachers and schools. We first describe the program. 



1.1 A Policy Inputs 

In program design, the grade-by*grade follow-through strategy is different from conventional practice. 
Because the State could target its expenditures in equipping a single year of classrooms, the concentration 
of computers potentially available per classroom was much higher than would be the case if the same num- 
ber of computers were distributed evenly, and thinly, across seven times as many classrooms (K-6). 

The investment in professional development also departs from the usual practice. West Virginia spent roughly 
30<t of every technology dollar on training, ten times the national average for schools. The State conforms to 
the recommendations of the U.S. Department of Education that 30 percent of the total technology budget be 
spent on professional development for teachers 2 . In one two-year period, 5,000 teachers were provided with 
professional development in a 'turn-key training' process provided by the State pursuant to a state contract 
with the software vendors. 

In contrast to the ordinary laissez-faire local selection of software, the state provided two sources of 
software— IBM or Jostens — between which local jurisdictions could choose. Jostens offered the Jostens 
Learning System in Reading and Mathematics, and IBM made available their Basic Skills Courseware. 
The Jostens Learning System provided identical software across schools; therefore, students who used the 
Jostens Learning System in one school were exposed to the same intervention as students in another school. 
The IBM offering, however, included a number of programs from which teachers could choose 3 . Each soft- 
ware package was developed or adapted by the vendor to emphasize the basic skills targeted by 
West Virginia for improvement — -reading and mathematics 4 . Each package was mapped onto West Virginia's 
statewide instructional goals and objectives. 

The option of two providers — IBM and Jostens Learning — allowed counties to select the solution that most 
closely aligned with the educational philosophy of the district. While both solutions meet the intent of the 
state program, they differ in their approach. Both solutions are strongly correlated to national standards and 
state instructional goals. Additionally, both were available to be implemented in classroom or lab configura- 
tions depending on the individual school's choice. 

IBM fosters a learning center approach in which students move in small groups to complete specific activi- 
ties at various classroom or lab locations designated as centers. Each session begins with a whole group 
activity and then moves to independent or cooperative use of the computer and courseware. According to 
IBM, this approach promotes problem-solving and higher-order thinking skills. The courseware offered by IBM 
addresses reading, mathematics and writing skills, while offering the districts some range of choices among 
software programs as teachers build a solution that complements and extends classroom instruction. 

2 US Department of Education, Getting America's Students Ready (or the 2 1st Century, Washington, DC, 1993. Most professional development 
took place during the school year. Teachers were paid their regular salaries and schools received paid substitutes. 

3 Bouncy Bee Learns Letters and Words, The Children's Writing and Publishing Center, Combining Sentences Series, Cornerstone Language and Math Series, Expl- 
oring Math Concepts Series, Exploring Math with Manipulatives Series, Exploring Measurement, Time, & Money Series. Math Concepts Series, Math and More. 
Math Practice Series, Parts of Speech Series, Primary Editor Plus, Punctuation Series, Reading tor Information Series, Reading for Meaning Series, Skillsbank 96 
Reading & Mathematics, Spelling Series, Stories and More l & II, Student Writing Center, Touch Typing for Beginners, Vocabulary Series, Writing to Read 2000, 
Writing to Write Series. 

4 The program as fielded in 1990-91, emphasized basic skills, and was targeted at the elementary grades. As a result, the software, especially the launch versions 
selected from the late 1980s available libraries, emphasized more drill and practice than would be the case in an initiative with access to current or future materials 
or one pointed at different policy goals. 
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Jostens Learning Corporation offers courseware that they believe addresses multiple instructional philoso- 
phies. The courseware allows the teacher to provide individualized instructional programs based on students' 
needs. The products are designed to enhance critical thinking skills through lessons for the reading, mathe- 
matics and writing curricula. The management system provides an assessment component with a variety of 
reporting options. 

In practice, the implementation of both vendor solutions has become very similar. Both have made a strong 
effort to adapt their materials to the needs of the teachers and students in West Virginia. Both vendors 
provided staff development specifically designed to address issues targeted by the BS/CE program. Both 
vendors provided correlation matrices to the texts on the West Virginia adoption lists and to the standard- 
ized assessment tool selected by the state. 

To fund this initiative, West Virginia used a per-student entitlement strategy rather than a grant strategy. Thus, 
all elementary schools received hardware, software and training, not just schools that successfully completed 
grant applications, as in many other jurisdictions. 

The eight-year consistency with which the legislature has passed appropriation bills to support BS/CE 
purchases is equally remarkable, Funding has been about $7 million per year. The state's policy choices 
concentrated resources on a relatively small number of classrooms and that, in turn, enhanced the likelihood 
of a ratio of students to computers that would reach a critical mass. Second, the state invested in its teach- 
ers as seriously as it did in technology. Third, it instituted policies that simplified supervision, training and 
maintenance (the single hardware contract and the two vendor choice for software). Fourth, all schools were 
included and all elementary teachers were assured of their place on the implementation schedule. Finally, 
the legislature provided consistent and sustained funding. The net effect of those choices was a tightly 
focused program. 

1.1 B What Is BS/CE to Students and Teachers? 

The reality of any program differs by the participant's perspective. 

1.1 B1 Students 

For most students, BS/CE represented only two of the three components of the initiative: hardware and soft- 
ware. The conventional wisdom is that change is resisted, but 90% of the state's students thought these new 
technologies in the school W'ere easy to use. Seventy-eight percent of West Virginia's fifth-grade students 
thought that the BS/CE computers had helped make them more successful as students. Two-thirds reported 
that they liked using computers "a lot"; more than half believe that computer technology is a “new basic" 
(two-thirds of the teachers have reached this conclusion). 

The fifth-grade students reported the amount of time they spent working on the computer each week, with 
the majority spending an hour or more a week; 10% of the students spend more than two hours a week. 
Nearly a quarter of the students (22%) work on the computer every day and an additional 36% spend time 
on the computer three or four days of the week. (See Figure 1 .) While BS/CE was intended to boost basic 
skills acquisition, the students inevitably learned about computers at the same time. After involvement in 
BS/CE, 43% report that they know a lot about computers. 
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Student attitudes and comfort with BS/CE and computers in general came from teacher encouragement, 
modeling, and support. Three-fourths of the students said they were encouraged by their teachers to 
use computers, 

Figure 1 - BS/CE Computer Use (Student Self-Reports) 
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1.1 B2 Teachers 

Most teachers understood BS/CE as a three part initiative: hardware, software, and technology develop 
ment, all directed at improving basic skills in students. The initiative sparked collaborative teaching practice, 
which was documented in our one-on-one interviews, case studies, and in the surveys. 

For instance, in West Virginia we saw BS/CE teachers opening classrooms early to accommodate students 
who wanted to use the rooms' computers. We saw other teachers taking pages of student test data off the 
printer and talking with each other about their differing perceptions of the students whose records were being 
printed. In another school, we interviewed two teachers from the same grade level who worked together 
using BS/CE because they believed that each of them was strongest in a different dimension. One created 
a series of math lessons customized to local traditions; the other wrote the beginning of a movie script that 
the students had to complete for a writing project. These activities were used by all of the students in the 
two classes. 
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Our procedures allowed us to determine what attitudes were most related to student achievement and that 
discussion extends the picture of BS/CE as it was perceived by the teachers. A few facts pre-figure those 
results: first, almost half the teachers (46%) report that their enthusiasm for incorporating computers into 
their instruction has increased over the years. (See Figure 2.) Ninety-two percent of the teachers concluded 
that instructional technology was not just another fad. 

Figure 2. Teacher Enthusiasm For BS/CE 



Increased 



Decreased 



Stayed the same 



Neither was the enthusiasm of the State's teachers diminished by the perceived requirement that they par- 
ticipate. Though BS/CE was technically not a mandate, when asked why they got involved, nearly half (49%) 
of the teachers said that initially they were "responding to a state-mandated requirement." But when we 
asked why they continued to be so involved, the percent reporting this 'compliance' motivation dropped to 
24%. Teachers came into BS/CE because they were compelled to, and they stayed because they believed 
in what it could do for them and their students. Fifty-eight percent of all teachers surveyed reported, "I feel 
competent guiding student learning through the use of Basic Skills computers." 

In addition to the enrichment and supplementary ways that teachers used BS/CE, they also used BS/CE as 
it was intended, for basic skills. In terms of the most to least frequent use by instructional purpose, teachers 
reported a mean of 4.40 for mathematics use and 4.06 for English/language arts use on a scale of 1 = None 
at all and 5 = Very much. Asked to estimate the amounts of time that students spent learning either mathe- 
matics or English/language arts with BS/CE computers, the majority of teachers chose the maximum time 
estimates possible. (See Table 1 .) 



Table 1 



Teacher Estimates of Student Computer Use by Curriculum Areas 




Teacher Estimate of Amount of Student Use 


Curriculum Area 


Percent 
Very Much 


Percent 

Some 


Percent 

Middle 


Percent 
a Little 


Percent 
None at All 


Mathematics 


62 


26 


9 


2 


i 


English/language arts 


57 


24 


16 


3 


1 
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One of the objections to computer-based learning is that it takes time away from higher order, more creative 
tasks. We asked the teachers whether they agreed or disagreed with this point of view: "When students 
learn using computers, they typically reproduce knowledge rather than construct it." One-fourth agreed, 
but 31% thought that computers were prompting more advanced knowledge construction in their classrooms. 

The same teachers described their own preferred roles as teachers in a more constructivist light ("coach" or 
"colleague" in the construction of knowledge) rather than in the traditional "source" or "master" of information. 

1. 1C Outcomes 

West Virginia's recent history of educational achievement is encouraging. Test performance improved over 
the BS/CE years. For example, after the technology enhanced cohort arrived in that grade, statewide third 
grade CTBS 5 scores went up five points. Prior to that time, those scores had risen about 1 .5 points per year, 
six points in four years. One interpretation of those trends is that the State's improvement trajectory was 
sustained and accelerated by BS/CE students. 

The BS/CE cohort's fourth-grade reading scores (1997) are reported to be the second highest among 
southern states (only one point behind North Carolina). On a national basis, if the achievement scores of 
various states are "corrected" by income, that is, if the unearned increment of school achievement that 
states with high per capita income enjoy from the support that privileged families give their children's learn- 
ing, then West Virginia's test scores improved more than those of any other state. In terms of per capita 
income, West Virginia is in 40th place: in achievement, it is in 17th place. Additionally, student attendance 
improved and early school leaving declined. 

The balance of this report details the relationship between the BS/CE initiative and the achievement out- 
comes which can be associated with that initiative. 



1 .2 Research Methods 

We know that these two events — a statewide, comprehensive instructional technology program 
and improved achievement scores — happened at the same time. This analysis tests the extent to which 
West Virginia's achievement gains are related to the BS/CE instructional technology initiative. 

The importance of research methods goes beyond academic concerns. Policymakers at every level have been 
investing in instructional technology and have a right to know, "Does it work?" And, at every level, different 
functions compete for scarce public dollars. Should money be spent to hire more para-professionals or to 
extend the school day? Should we build new facilities or wire existing ones? Research evidence does not 
"make decisions": policy is shaped by the interests of various publics, by compromises and accommodations 
and by the momentum of existing arrangements. The evidence of research is only one part of decision 
making but, in its absence, decisions are made solely on grounds external to what best helps children learn. 



5 CTBS, California Test of Basic Skills. 
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Thus, the Milken Exchange on Education Technology commissioned research that would have several 
policy relevant characteristics. First, because states have the reserved power to determine schooling policy, 
they wanted statewide evidence. Second, because test scores are one central metric of school outcomes 
and children's accomplishments, they required an analysis of educational achievement, using conventional 
state level assessments. Third, because of recurrent skepticism about instructional technology 6 , a well con- 
structed analysis of results was needed to inform the public discourse and professional practice. 

The hallmarks of a good evaluation design include: a sample selected to represent the population being 
studied; data that are valid, reliable, and from multiple perspectives; students as the unit of analysis; 
multiple forms of implementation documentation; and, the continuing cooperation of the group studied. 

The first step in the analysis was a series of meetings to establish the feasibility of West Virginia as a study 
site. With the cooperation of key state leaders secured and with access to data assured, Interactive, Inc. was 
able to select a sample, gain entry to schools to administer paper and pencil surveys to all fifth-grade 
students and all teachers; interview all fifth grade teachers, all principals, and selected students; and attach 
Stanford 9 achievement data for two years to all students in the sample. 

1.2A Sample 

Since the purpose of this study is to examine the link between technology and student achievement, the unit 
of analysis is the student. Because we need to document specific implementation, teacher attitudes, and 
other variables in relation to each student we studied, a random selection of students statewide was not fea- 
sible within the time and cost restraints of the study. Additionally, because this is a retrospective longitudinal 
study which collects up to five years of data on schools, classrooms, and students, we needed to use our 
resources to insure as much depth and accuracy in these data as possible. Resource considerations pro- 
hibited both local data collection in every school in the state (or even a random selection of a representative 
sample of schools) and a random sample of students, since the latter choice would require the research team 
to collect data in each student's school. Thus, we used schools as an initial stratifier for the sample. 

In order to assure that the schools selected would provide a representative sample of West Virginia BS/CE 
students, we selected 18 schools as the initial stratifier from which we would study all students in those 
schools. The schools were selected with the help of a state education advisory group based upon achieve- 
ment, perceived BS/CE intensivity, geography, vendors, and SES. 

Based on 1995-96 3rd-grade CTBS scores, the 18 schools selected for study range from high to low in 
school level achievement. Furthermore, achievement varies naturally among students and that is appropriate 
since the unit of analysis here is the student. 

Our next concern in sample selection was to find varying student technology experiences. Consensus among 
West Virginia state officials, software vendor consultants with regular access to the schools, and current 
research on student technology experience is that student technology experiences vary as much within group 
(across students in a school) as among groups (across schools). Therefore, selecting schools based on high 
vs. low technology is not necessary to insure a wide variance in experience and attitudes. 

6 The skepticism is often a by-product of the competition for scarce resources, in the struggle lo get budgets for their preferences, the partisans of any initiative will 
question the evidence on which the competition bases its claims. 
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However, as an additional safeguard, we chose schools which, based upon the judgements of both 
West Virginia Education officials and the software consultants who are in those schools regularly, were 
perceived to have a range from most to least technology practice. 

Another criterion in the selection of the schools was geography. State officials believed this an important vari- 
able for sample inclusion — both in terms of variance in achievement scores and in local factors which may 
have an impact on technology use. As a result of discussions with the state education advisory group, we 
divided the state into four geographic areas which represent distinct geocultural and educational variables: 
Northern panhandle (adjacent to Ohio and Pennsylvania), Eastern panhandle (essentially a suburb of 
Washington D.C.), South (rural and adjacent to Kentucky and Virginia), and South Central (the capital, 
Charleston and its surrounding counties). 

An additional and convenient geographic stratifier in selecting the sample is the eight Regional Educational Service 
Agencies (RESA). The eighteen schools for study come from all four geographic areas and all eight RESAs. 

Part of the policy significance of this analysis is West Virginia's unique statewide implementation strategy 
which required a choice between two software vendors: IBM and Jostens Learning Center. Additionally, the 
majority of the implementation assistance came from the software vendor consultants, not the state. 
Therefore, software is an important consideration in reconstructing implementation and individual experi- 
ences. We selected schools so that the proportion of Jostens and IBM programs used by students would 
be proportional to their use statewide. As a result, twelve of the schools selected were Jostens schools and 
six were IBM schools. 

Based on West Virginia Department of Education data on the percentage of students receiving free and 
reduced lunch, the 18 schools also range from high to low in terms of socioeconomic status. 

The result is a sample of 18 schools which vary with respect to: 

^0 achievement test scores, 

BS/CE and technology experience, 

^0 geography, 

^0 software vendors, and 

^0 community SES. 

The next step in the sample selection was to determine which students from the 18 schools would be stud- 
ied. Prior to the 1 996-97 school year in West Virginia, the CTBS was administered for the 3rd-, 6th-, 9th- and 
1 1th-grade students only. Beginning in the 1 996-97 school year, the Stanford Achievement Test (9th Edition) 
was administered to all students in grades 3 through 11. Therefore, the current 5th-grade students are the 
only students in the state of West Virginia for which the state has three consecutive years of achievement 
test data and who have had any BS/CE exposure. Because of BS/CE's follow-through phasing, the current 
5th graders are the students for whom technology has been most available. Therefore, we documented the 
experiences of all current 5th-grade students from their first year of experience with BS/CE through their 
current experience. 

Our student sample included all 950 students in 5th grade in the 18 stratified schools. To generalize to all 
K-6 students In West Virginia from K-6 (N=1 61 ,231 ) at a 95% level of confidence, we would have needed 
data on 384 students. This sample, then, is representative both in that it includes students with a range of 
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achievement, technology experiences, geography, and SES, and it is nearly three times more students than 
necessary to assure confidence in the accuracy of the analysis. 

1 ,2B Data Collection: Inputs 

Because this study is an examination of the relationship between technology use and student achievement, 
we had to determine how we would document technology use as well as other variables which might influ- 
ence use. The technology factors included: BS/CE use; other technology use; attitudes toward technology; 
home technology practices; and other demographic and school variables. 

To determine student use and attitudes, we used student and teacher survey and interview data. Input data 
were collected in the following ways. 

•ft Surveys of 950 5th-grade students, using a 33-item paper and pencil survey. This survey focused 
on BS/CE use; attitudes toward schools, computers, and technology in general; and other 
factors which relate to technology use and learning. This survey allowed students to record their 
technology experiences from kindergarten through their current school year. 

ft Surveys of 290 third- through fifth-grade teachers, using a 99 item paper and pencil survey. 
We administered this survey to all the teachers in all 18 schools in an attempt to capture data from 
those teachers who had taught our fifth-grade student sample for the past three years. 

-ft Interviews with all fifth-grade teachers in the 1 8 sample schools. 

~ft Interviews with all principals in the 18 sample schools. 

"ft Interviews with selected early-grade teachers in the sample schools. 

-ft Analysis of documents in each school related to technology planning and implementation. 

In addition to the data from students, we asked teachers about student use as well as their own attitudes 
and practices. Ninety-one percent of the teachers were female. Sixty-three percent have been teaching for 
more than 21 years; 29% for 1 3-20 years; 5% for 6-1 2 years; and only 4% have been teaching for 5 or fewer 
years. 

We interviewed all the fifth-grade teachers and selected early-grade teachers in each school to try and under- 
stand more completely how BS/CE was implemented in their classrooms and what this meant for how 
students learn. These teachers provided current use data for each student on BS/CE and their own past 
curriculum use of BS/CE. 

During the winter of 1998, each school was documented, on site, by an Interactive, Inc. ‘field researcher. To 
determine if other initiatives were happening or if political or internal issues might have affected both imple- 
mentation of BS/CE and student involvement, we interviewed all principals, fifth-grade teachers, and selected 
early-grade teachers in the 18 schools. In addition, we analyzed classroom, school, and state documents to 
complete the case descriptions. 

1.2C Data Collection: Outputs 

Once we collected the student use data and the BS/CE descriptions, we turned to measures of achieve- 
ment. Prior to 1996, the West Virginia Department of Education administered the CTBS to all students in 
grades three, six, nine and eleven. Beginning in the 1996-97 school year, a new statewide assessment pro- 
gram was implemented with all students in grades 3-1 1 taking the grade-appropriate Stanford-9 achievement 
test. Thus, we had two years of Stanford-9 test score data to consider, and an opportunity to consider the 
relationship between gain scores and BS/CE. 
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The scores that were used in the analysis were scaled scores on the Stanford-9 achievement test. Because 
scaled scores are normed against a nationally representative group, they are appropriate for comparison pur- 
poses and for the computation of gain scores. The Stanford-9 achievement test is divided into a number of 
subtests, and where appropriate, those scores are reported. Our focus is on gains in "basic skills," a score 
computed and reported by the West Virginia Department of - Education for comparison purposes, and a 
score that measures the math, reading and language arts areas that were the focus of BS/CE. Scores 
on each of the three areas could range from 400 to 800. For 1998, the range of "basic skills scores in the 
state was from 547 to 766 and the average gain score was 14 points. 

In this study, we computed gains in Stanford-9 results to measure student achievement from one year to the 
next. Gain scores represented the difference between the scores on a basic skills combined measure of math 
and reading and language arts from the 1996-97 test to the 1997-98 test. Thus, achievement was repre- 
sented by Stanford-9 scores on reading, language arts and mathematics for all students in the sample for 
1997-98 and 1996-97 

1 .2D The Model: Access/Attitude/Training 

To determine the relationship between the BS/CE inputs and the achievement outputs, we analyzed our data 
using a model that includes access to software and computers, attitudes toward technology, and teacher 
training and involvement: 



The Model: Access/Attitude/Training 

(1) Software and Computer Availability and Use (software focused on basic skills, computer 
availability, availability of other software and technology programs, time using basic skills software) 

+ (2) Attitudes Toward Computers (student attitudes toward computers, teacher attitudes toward computers) 

+ (3) Teacher Training and Involvement in Technology Implementation Decisions (teacher profes- 

sional development and involvement in implementation decisions) 

= Predicted Change in Acheivement Test Scores 

We speculated that Stanford 9 achievement gains would be the greatest in schools with the highest amounts 
of the model components. 

1.2D1 Software and Computer Availability and Use 

This component includes four items: basic skills software, hardware availability, availability of other 
software and technology, and time spent using basic skills software. 

(1) Basic Skills Software. The development of interactive software to teach basic skills at each grade level 
and the availability of that software to students and teachers is likely to result in student use of the software. 
Data for this variable comes from observation and the West Virginia Department of Education. 

(2) Computer Availability. The more computers that are available to students and the more students report 
that there are computers available when they want them, the more likely it is that student computer use and 
the technology implementation can occur. Data for this variable comes from a yes/no response on the 
student survey. 
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(3) Availability of Other Software and Technology. This is a school wide variable which was measured in the 
site visits by inquiry from principals, teachers, and students as well as by observation of technology initia- 
tives and the collection of documents submitted to the State Department of Education. 

(4) Time Using Software. This component of the model includes both time spent on B^/CE and time spent 
using other software. We asked teachers to describe their students' use of computers and BS/CE. Their 
answers resulted in a factor which combined the responses to six questions in the teacher survey and 
represents the amount of time that teachers estimate students use the technology in math, reading, and writ- 
ing. Total scores on this factor ranged from 5 to 15, moving from none in math, reading, and writing to 
60 minutes or more in math, reading, and writing. 

A series of questions to students resulted in a similar student-reported technology use factor. Teacher and 
student measures of time were closely correlated. Total scores on this 18 point factor ranged from 4 to 18, 
from zero in math, reading, and writing to 60 minutes or more in math, reading, and writing. (See Figure 3.) 



Figure 3. Time Using BS/CE Computers 





5th Grade Use Factor 5th Grade Use Factor 

(Based on Student Estimates) (Based on Teacher Estimates) 

The second "use” component focused on technology use other than BS/CE. We predicted that time using 
any instructional technology leads to increased skill and comfort with computers and the technology and 
basic skill learning of the primary initiative. The use of technology independent of BS/CE was documented in 
the qualitative case studies in each school and each classroom. The case study data were then recoded into 
a four option variable ranging from "no additional technology" to "extensive additional technology." 
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1.2D2 Attitudes Toward Computers 

The attitude component includes both student and teacher measures. 

The student attitude factor combined the results of student responses to two survey questions: the scores 
ranged from two to four. The model predicts that the more positive students feel about computers and tech- 
nology, the more likely they are to use BS/CE software. 

Two factors represented teacher attitudes. The first was a factor combining the results of questions from the 
teacher survey focusing on teachers' comfort level and confidence in using computers. The second is a 
factor that combined the responses to two questions which tapped teacher willingness to continue learning 
about computers and technology. 

1.2D3 Teacher Professional Development and Involvement in Implementation Decisions 

This component includes both the training that teachers received and their involvement in the planning and 
impleme itation of BS/CE. 

Teacher training is the amount of time teachers spent learning on the computer on their own. We predicted 
that the more teachers are involved in development of the technolugy implementation plan, the more likely 
the implementation is to be carried out. We measured this involvement using questions on the teacher 
survey. Teachers responded to a five point Likert scale. The results were clarified through teacher interviews. 

1.2E Analysis 

After factor analysis had established the reliability and validity of the three components just described, the 
relationship of BS/CE to student achievement gain scores on the Stanford-9 was analyzed through multiple 
regression analysis. 

Interactive, Inc. circulated all reports in draft to officials from the West Virginia Department of Education for 
correction of matters of fact and for comments on our interpretations and conclusions. [ Nota Bene : 
responsibility for this analysis, its conclusions and recommendations rests solely with the authors 
and does not reflect the policies of either the West Virginia Department of Education or the Milken Exchange 
on Education Technology.] 



1.3 Summary 

The net result of the state's initiative, its accomplishments and the procedures for this analysis makes 
West Virginia a potentially illuminating case study of innovation in instructional technology. The implementa- 
tion was statewide yet tightly focused. The outcomes were significant in terms of policy and of statistics. And 
the events and the outcomes have been comprehensively assessed. 
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2.0 RELATIONSHIP OF BS/CE TO STUDENT ACHIEVEMENT 



To examine the relationship between the BS/CE experience and student achievement, we computed gain 
scores on the Stanford-9 for each student from 1996-9.7 to 1997-98. Additionally, for each student we 
gathered data for each of the regression model components which measure: 

^b Software and Computer Availability and Use, 

Attitudes Toward Computers, and 

^b Teacher Professional Development and Involvement in Technology Basic Skills Implementation Decisions. 



2.1 BS/CE's Effect on Student Achievement 

i 

With the student as the unit of analysis, we examined the relationship between how much of each of the 
model variables that student had experienced and her or his gain scores on the Stanford-9. 

The more of each of the model components that the student experienced, the higher the gain score on the 
Stanford-9. Specifically, the BS/CE technology regression model accounts for 11% of the total variance in 
the basic skills 7 on the Stanford-9 achievement test. (See Table 2.) 



Table 2 



Model Summary: Achievement gain scores of the fifth-grade students. 


r 


r 2 


adjusted r 2 


std error of 
the estimate 


.331 


,ii 


.094 


14.8317 



Thus, student gain scores can be partially explained by a model composed of factors that describe the 
overall BS/CE experience, an impact which is powerful and practically significant; it is also statistically 
significant at more than the .001 level. More to a practical point, an 1 1% improvement in test scores would 
be welcomed by most students, parents and educators. 



2.2 Policy Significance of the BS/CE Effect on Student Achievement 

Analysis indicates that test scores of our student sample improved from 1996-97 to 1997-98. Conventional 
procedures ask, "What percentage of that whole gain can be explained by the BS/CE initiative?" Our data 
indicate that 1 1% of the reason why student scores increased was because of BS/CE. 

What does this mean? Is 11% more than just statistically significant? Does it have any practical value to 
students, educators and policymakers? We think so. To understand what the 11% of explained variance 
means, it is important to understand all the influences on student achievement, the most significant being 
family background. 




.'27. 



' An average of the students' Total Math, Total Reading and Language scores 
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Family background explains a great deal of why some children do better than others. Since James Coleman's 
1965 research, we have known that children's achievement in school is conditioned more by what they bring 
to school than by what schools are able to do with them. Consider: 

(D)espite the wide range of diversity of school facilities, curriculum and teachers, and despite the wide 
diversity among student bodies in different schools, over 70% of the variation in achievement for each 
group is variation within the same student body (James S. Coleman, et al, Equality of Educational 
Opportunity. Washington DC., US Department of Health, Education and Welfare, 1966, p 77). 

In thinking about schools, this difference between what schools can and cannot influence is intuitively and 
practically obvious. Imagine trying to search out the explanations for gains in test scores between two groups 
of children. One group lives in houses where the following are customary: daily reading that is modeled by 
adults and encouraged for children; availability of books and magazines; frequent use of computers and com- 
puter games; after school learning and enrichment activities; parental or other adult supervision of homework; 
and, visits to museums, art galleries, and science sites. 

The other group of children live in homes where: there are no books; reading is not modeled, encouraged 
or frequent; unsupervised television is constant; there is no computer; there is no after school enrichment; 
there is no supervision with homework; and, there are no visits to cultural events of the sort reflected on 
achievefnent tests. 

To document the amount of learning for both groups of children, an achievement test is administered which 
measures: knowledge of mainstream culture; familiarity with standard English; art and science experiences 
outside the classroom; and the benefit of extra drill and practice in the basic skills. 

It is not surprising that the children with all of the educational and cultural extras provided by the family and 
tested by the assessment do better than the children who do not have the related experiences. This is the 
power of the family and of economic privilege. According to Coleman, the resources of families, the culture, 
the media and the peer group account for 70% of the differences in student achievement. 

If the performance of children depends heavily on the characteristics of the families they come from and if 
we further understand that those family characteristics are for the most part beyond the reach of public 
policy (for example, proscribing divorce, prescribing post-graduate education for parents, and providing all 
families with equal social capital), then we ought to concentrate our attention on the things that are, in fact, 
accessible to influence by school policy. 

If, as Coleman and others assert, 70% of the variation in test score performance relates to family and other back- 
ground factors, that leaves 30% that schools can influence. We call that 30%, w school accessible performance ' 0 . 

This analysis documents that as much as 11% of the gain score variance for one year can be explained 
by BS/CE. Thus, BS/CE explains more than a third of the school reasons why students' achievement 
scores improved. Said another way, of all the factors that can affect a child's learning, about 30% is 
within the school's sphere of influence. The BS/CE technology initiative explained nearly 11 of those 30 
percentage points. 




9 The school improvement literature concentrates on the institutional side of this equation, not on the larger family and community surround. Ideas of ’school effects’ 
and of ’within school factors' can be found, for example in Michael Rutter, Barbara Maughan, Janet Ouston, Alan Smith, 15,000 Hours: Secondary Schools 
and Their Ellecls on Children , Cambridge, Harvard University Press, 1979. See also, Peter Mortimore, Pamela Sammons, Louise Stoll, David Lewis, Russel 
Ecob, School Matters: The Junior Years , Somerset, Open Books, 1988. 
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This effect is reflected in the change in West Virginia's rankings among other states. During the period 
studied, West Virginia moved up the list of states in the nation rank-ordered by school achievement from 
thirty-third to seventeenth best. Since we can establish the fraction of achievement score gain that is related 
to BS/CE, it is reasonable to assign some portion of the credit to that initiative. It is also reasonable to 
believe that the effects of BS/CE, although impressive, are underestimated. 

There are two additional reasons to believe that the reported gain scores underestimate the total effect of 
BS/CE. The 11% of the gain score variation explained by BS/CE was between 1997 and 1998 scores. 
Those gains happened between the fourth and fifth grades and it is very likely that there were earlier increas- 
es, attributable to this initiative, between the third and fourth grades and the second and third grades and so 
on back to the beginning. Thus, the cumulative effect of BS/CE most likely accounts for more than the 11% 
of the single-year gains that we found. 

We also suspect that the gain scores reported are unnecessarily conservative because current hardware and 
software is more powerful than what was implemented in BS/CE's early years. Current technology is likely 
to yield even larger gains. 

If BS/CE can be credited with some considerable effect on statewide test score gains, then did BS/CE 
"make children perform better?" The question is reasonable but on the evidence we have, strictly interpret- 
ed within the canons of social science, we cannot say with certainty. To be able to do so, the State would 
have had to have randomly assigned some students to BS/CE and randomly assigned other students to 
conditions where BS/CE was not available. While that might be good social science, such practices are poor 
and probably unethical public and educational policy. 

Thus, as a consequence of the statewide implementation, it was not possible for us to compare control group 
schools with the BS/CE schools. And, the achievement score record prior to 1990 is insufficiently detailed 
to support a pre/post inquiry. As a result, our's is an analysis of correlations (things that vary together), not an 
analysis of causation 9 . No one withheld instructional technology from half the children; instead the State 
made BS/CE available to all children. However, as is usual in any initiative, some students experienced more 
of each of the BS/CE technology components than others, so what we have is a situation where ail students 
received something and some students received more of some things than did other students. Those 
combinations are not uniform and vary by student, even within the same school or the same classroom. And 
those variations make possible the procedures of this inquiry. 



As important as the procedures of social science research are, the needs of policymakers are also legitimate, 
so it is possible and sometimes helpful to interpret correlational data to suggest causality 10 . We have done 
so in this instance, suggesting that the unstandardized betas that we report are indices of the effect of the 
related BS/CE component on Stanford-9 gain scores. (See Section 2.3.) 

We can help with the "What works" question by first, assessing the statistical significance. of the numerical 
findings. Big findings may suggest big relationships. Second, we can ask about other explanations for 
the findings. Sometimes those alternate explanations can be disposed of; sometimes they can be understood. 

9 The absence of findings that rise to the level of causation is the norm in social policy. Virtually no decision in public schooling is informed by causal analysis 
—not finance decisions, not racial integration decisions, not curriculum or testing or personnel selection decisions. 

10 Pedhazur and Schmelkin (1996) believe that the unstandardized betas in nonexperimental research can be interpreted as indices of effect if the model is specified 
based upon theoretical foundations and previous research. Because our model is so specified, we maintain that our unstandardized betas indicate cffecl. 
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In this case, we believe that the several dimensions of BS/CE explain a substantial fraction of the school- 
accessible performance. An explanation of that magnitude is intrinsically worthy. The part that is not explained 
by BS/CE may have to do with initiatives such as statewide standards and testing, or West Virginia's "Unified 
School Improvement Plan(ning)" or school-site accreditation visits coupled to the possibility of probation. 
Empirically determining the effect of those procedures was not within the scope of this work, but we did 
examine nearly 50 other variables for alternative explanations and found none 11 . (See also "4.0 Additional 
Explanations For Test Score Gains.") 

We believe that reasonable people, experienced with school policy, will find that this analysis supports the conclu- 
sion that instructional technology is a powerful assist to children's achievement 12 and thus to school improvement 



2.3 Effects of the Components of BS/CE on Achievement 



Pubiic policy has a continuing interest in answers to the "What Works?" question. Here, the whole BS/CE 
program makes a difference. No single component is sufficient to account for the BS/CE-associated 
outcomes and neither, probably, would it have been sufficient to maintain this program for only a few years. 

It is not surprising that no single component dominates. Putting hardware in a room without training teachers 
or otherwise supporting the integration of technology into the classroom cannot be expected to make a 
difference. It is the cumulative effect of the several variables that compose the model that is important. 

The several factors of our model resemble the several components of BS/CE policy. Statistically, as practi- 
cally, it takes multiple dimensions to make a difference. The model demonstrates that it is software specific 
to the purposes of basic skills achievement, availability of computers, teacher training and involvement in 
implementation decisions, positive student and teacher attitudes toward computers, and time spent using the 
software that lead to achievement gains. 

Taken together, these factors account for 1 1% of the variance in basic skills gain scores at more than a .001 
confidence level (n = 502). 

The three components of this empirically-derived model — access, attitude, and training — are similar to what 
leaders in instructional technology advocate. Unless there is sufficient equipment and opportunity to learn 
(access) there are unlikely to be effects from instructional technology. And, unless teachers have a chance 
to learn about the technology and how it may make them and their students more successful (training), they 
are less likely to believe It can help (attitude) and, in fact, to use it. 



11 For instance, we examined the effects of several student, teacher, and school variables on gainscores in our model crealion and testing and found no additional 
explanatory power, for the anatytic modet, from these variables. Student variables tested inctude race, sex, age, geography, homework support, familiarity with 

, Internet and E-mail, attitudes about school, attitudes about learning, time spent on homework, grades, and other achievement measures; teacher variables include 
attitudes, pedagogical philosophy, experience with computers and software, attitudes toward state initiatives, attitudes about teaching, attitudes about student 
learning,’ cost to teacher (in time, etc.) of implementation, professional development, homework expectations, and attitudes about families; district variables include 
local political actions, changes in administration, additional initiatives, deployment methods, availability of labs, and colleagueship. 

12 54% of the teachers thought that BS/CE was Instrumental in test score gains. 
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The teachers' confidence in using computers and the amount of time they spend using them are a bundle of 
attributes that may be influenced by experience, professional development and the teachers' perception of 
their principal's leadership, The teachers' attitudes and behaviors also interact with what the students' 
experience — increased use and positive attitudes toward computers. 

We constructed the model first to test the possible relationships between BS/CE and student achievement 
and second to illuminate how BS/CE is probably making a difference with students. Empirically, we know that 
the complex of factors listed above will explain the amounts of achievement gain we have reported. 

Practically, it is likely that BS/CE is having its effects through its ability to impact students through the model 
factors. With students, for example, the availability of computers and teacher encouragement are likely to 
increase computer use and, as use increases so does the amount learned and thus (probably) achievement. 
Similar logic applies to teachers. As BS/CE has rippled through the faculty culture over the years, it has 
changed attitudes and that becomes part of the intervention, part of the effect and part of the success. 

For example, 69% of the students thought computers were as important as "reading and writing." Sixty-six 
percent like computers a lot, only 5% do not like computers. Sixty-one percent report that their teachers 
encouraged them to use computers. 

School reform is often criticized for relying on "The Magic Feather Principle;" the idea that there will be a 
singular solution to complex problems. West Virginia's BS/CE model used multiple interventions to support 
and change multiple functions of teaching and learning. 



2.4 Access/Attitude/Training Model Effect Size 



As noted in section 1.2D, there are three major components of the technology initiative under study: 

^ Software and Computer Availability and Use (software focused on basic skills, time using BS/CS, 
use of other computer technology initiatives, availability of computers), 

^ Attitudes Toward Computers (student attitudes toward computers, teacher attitudes toward computers), 
^ Teacher Training and Involvement in BS/CE Implementation Decisions. (See Table 3.) 

Understanding the strength of each of the model components is best done by examining the betas. While 
betas are a kind of effect size, they are not a straightforward measure, especially in the social sciences where 
variables are often Inter-correlated. However, where a model is well specified, even if the study is not exper- 
imental, it is possible, particularly for policy purposes, to interpret unstandardized betas as indices of effects. 

In our case, all of the components of this model stand as independent variables because there is very little 
correlation among and between them. The correlation between variables ranges from a low of ,079 to a high 
of .224 and, thus the model has relatively low multicollinearity among its components. The tolerance for each 
variable In this model runs from .564 to .971, where the value is the proportion of variance unique to 
each variable. Six of the nine variables report tolerances in the 80th and 90th percentiles, while three hover 
at about the 60th percentile. 
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Table 3 

Acceaa/Attltude/Tralnlng Modal Effect Sizes 



Unstandanfizad Cbafflclonta StandarthuKl Coamdonta 



Hardware and Software Access and Use 




B 


Std. Error 


Beta 


t 


Sig 


Computer and 
BS/CE Use 


1.453 


.272 


.240 


5.338 


.000 


Software Access 


5.958 


1.470 


.184 


4.054 


.000 j 


Computer Access 


2.126 


1.347 


.068 


1.579 


.115 


Other Technology 
Access 


.715 


.659 


.051 


1.086 


.278 




Attitudes 


Student Attitudes 
Towards Computers 


2.742 


1.222 


.099 


2.243 


.025 


Teacher Attitudes 
Towards Computers 


1.069 


.434 


.123 


2.461 


.014 


Teacher Confidence 


.310 


.584 


.030 


.530 


.596 1 


Teacher Training and Invotvment 


Involvement 


1 -1.959' 


1.059 


1 -.097 




•1.851 


.065 


Training 


1.109 


.656 


.095 




1.619 


.091 



'NagaUva numbers Indcetad Wflher teacher Involvement 



The meaning of the unstandardized betas tells us how much of an effect the nine variables that make up the 
three components of our model have on achievement, which In our case means that If all the components of 
the model are In place, achievement Increases by between eight and ten months. 

In schools that are hard-pressed to find time for all the mandates and all the required classes, time counts. 
In this analysis, "time" has two facets— the frequency with which BS/CE was used in any given year and the 
repeated, cumulated experience of BS/CE over the years. 

The variable that measures reported instructional time in a year accounts for more of the difference than any 
other. The relation between time-on-task and learning gains Is commonly observed. Extending the school day 
or year, cutting recess, dropping curriculum topics all make possible Increases In tlme-on-task. What Is 
different here Is the use of technology to Increase tlme-on-task. In a direct Instruction, teacher-centered 
classroom there are a fixed number of minutes for a teacher to deliver Instruction. When students use 
computers to work independently, the amount of student tlme-on-task can be multiplied Independent of the 
limits on the teacher's agenda and availability. 

Teachers make decisions about classroom time but so do state policymakers. BS/CE has been sustained 
over an uncommonly long Interval, seven years. Students reported Increased computer use every year from 
kindergarten through the fourth grade. For example, the percentage of students who described themselves 
as using ’BS/CE computers a lot" Increased from 20% in kindergarten to 53% m the fourth grade. Similarly, 
the earlier in their school careers that students began using computers, the more likely they were to 
continue Intensive use. (See Figure 4.) 

best copy available 
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Figure 4 . B5/CE Year-to-Year Computer Use 




None 



Some 




A lot 



And, almost half the teachers became more enthusiastic about BS/CE as time passed. Neither would have 
happened without the willingness of the state's policymakers to keep instructional technology at or near the 
top of the priority list. We believe that time within the school day and year AND time across the student's life 
in school are part of the context for the achievement score gains. 
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2.5 The Distribution of Achievement Gains Across Groups of Children 

The benefits of BS/CE fall differently by type of child. Gender and SES effect how well the model predicts 
achievement. 

2.5A Growth for All Students 

The amount of the gain score changes explained by BS/CE are significant in both policy and statistical terms. 
But, gain score differences focus the amount of change in one child or group compared to the amount of 
change in another child or group. What they do not reveal is whether or not the whole group was getting bet 
ter and, in West Virginia, it was. in addition, students were able to recognize the relationship between their 
computer experience on BS/CE and their learning gains: Seventy-eight percent of the fifth graders thought 
that computers had helped make them better students. 

The effect of the BS/CE technology initiative on student achievement can be illustrated by thinking about 
a bank of elevators. All the elevators are carrying passengers up, but their speeds may vary. In addition, pas- 
sengers get on at different floors and get off at different floors. All passengers get someplace in the elevator, 
they start at different places and stop at different places, and they all go up. It is important to see that, although 
they start in different places and some get to the top faster than others, all the passengers are being lifted. 

2.5B Equity Effects 

The data indicate that BS/CE helped all children learn aM it helped the neediest children the most! (Special 
education students reported the same amount of time on BS/CE computers as regular education students). 
Those children without computers at home made the biggest gains and that is good news for public policy 
the children who most need the public school can be helped by this sort of policy. The schools we studied 
were chosen to represent a range of socio-economic characteristics. Some were urban, suburban and rural. 
Some schools served decidedly higher income families than did others. 

The "Digital Divide" is mapped by those West Virginia students who do (62%) and do not (38%) have 
computers at home. Those without compute rs at home gained more in. 

total basic skills, 

^ total language, 

''6 language expression, 
v '& total reading, 
v/ 5 reading comprehension, and 
vocabulary. 

The Arcess/AttiturWTraininn Model explains more of the basic skills gain scores for students who report 
lower grades than for students who report higher grades. For students who report receiving grades of C, the 
model explains 1 9.3%, versus 1 5.6% for students who report grades of B, and 10.7% for those reporting As. 

Thus, BS/CE Is more strongly related to gains for students who have less family and social capital and for 
students who do less well in school. 
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The chronic challenges of race and education condition us to expect lower achievement for some children 
than others. It is progress that in West Virginia, there were no differences in gain scores between white 
students and non-white students. On 1998 achievement, there were no differences overall, but white 
students achieved higher scores on listening, vocabulary and reading. 



2.5C Gender 

One regularity of both the technology and the gender literatures is the relative disadvantage of girls with 
respect to technology. It is progress that girls and boys had the same access and the same use of comput- 
ers in West Virginia. 

While there are no differences in the amount of use between girls and boys, the girls were more likely to see 
computers as a tool and the boys as a toy; boys are more likely to report that computers are fun. 

Equal access and use of computers and software by girls and boys in West Virginia proved important in terms 
of girls' comfort with and attitudes toward computers. The more years that girls report having used com- 
puters, the more they like them and the more they report knowing about them. Unlike many girls, the girls in 
West Virginia are more likely than the boys to say they know more about computers than do boys. In addi- 
tion, the girls reported finding computers more accessible than their teachers to their particular learning 
needs; girls are more likely to consider it easier to learn from computers than from their teacher. This finding 
might indicate that computers, unlike some teachers, respond in the same ways to both girls and boys and 
that either sex can ask questions, linger, or repeat activities on a computer. 

In terms of gain scores, there were differences in only two areas related to gender — girls gained more in 
social stud 1 s and boys gained more in spelling. In math and reading, there were no gender differences. 
However, for the actual 1 997-98 scores, girls did better in language, reading and study skills and boys did 
better in spelling. There were no gender differences in mathematics in 1998. 

The Access/ Attiturie/Traininq Model is a more powerful predictor for boys than for girls. The model explains 
16.1% of the basic skills gain score for boys, compared to 11% for girls. This is probably because of the dif- 
ference in 1998 achievement on the language subtests on the Stanford 9. Because girls did better than boys 
on those language subtests, and because language and reading were improved by using the BS/CE tech- 
nology, the more boys used BS/CE, the more likely they were to improve on Stanford 9 measure of language. 

2.6 Teachers and BS/CE 

Fifty-one percent of West Virginia's teachers are confident in using computers in their teaching, only 1 9% are 
not. Two-thirds say they are very Interested In computers: a third believe that technology has "empowered" 
them as teachers. Half the teachers thought that technology had helped "a lot with West Virginia Instructional 
goals and objectives." 

We asked teachers to grade themselves according to how skillful they were in using computers for various 
functions. Typically, the closer the activity is to the "performance art" core of classroom work, the lower the 
marks teachers give themselves. While West Virginia teachers select word processing as their most 
skillful application, the next highest rated functions are central to teaching. The table below Includes com- 
parison values from another large-scale analysis of teacher use of technology. 
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The skill levels in delivering instruction, planning lessons, managing paperwork, and word processing of 
West Virginia's teachers who had computers in their classrooms were significantly higher than those who had 
computers in the labs, (See Table 4.) 



Table 4 



Teacher Self-Estimates of Skill Levels far Various Functions 

(l\l = 290] 


Function 


Mean Score* 
West Virginia 


Mean Score 
New York 


Delivering Instruction 


3.00 


2.48 


Planning Lessons 


2.52 


2.48 


Networking 


2.42 


2.04 



‘Range: 1 = low, 5 = high, The New York data are from: Dale Mann and Edward A. Shafer, “Technology and 
Achievement" The American School Board Journal, v 184, n 7, July 1997, pp 22-23 based upon a 1996 study done 
for the Mohawk Regional Information Center, Verona, New York by Charol Shakeshaft, Dale Mann, Robert Kottkamp, 
and Mike Mussi, Interactive, Inc,, Huntington, New York. 



Simple correlations suggest that collaborative learning and constructivist ideas were most likely to occur in 
the classrooms of those teachers who report the highest computer confidence level and the most time using 
computers in their instruction (both factors that contribute to the BS/CE Access/ Attitude/Training Model). 

We asked some questions to test possible negative attitudes about computers. (See Table 5.) 

Table 5 



Teacher Reactions to Purj 
Computer-Reli 


jorted Negative Effc 
ited Technology 4 ' 


icts from 


Purported Effect 


% Disagree 


% Agree 


"Computers... 






...take time from direct instruction" 


61 


19 


...make class management more difficult" 


65 


13 


...promote frustration and aggravation” 


51 


20 


...require more planning" 


39 


28 


...take too much time to use" 63 H 



‘Mid-point, "3," responses not reported. 



The most frequently chosen comment— that enhancing instruction by using computers 
requires more planning time — reflects the reality that adding new techniques takes time. 
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3.P IMPLEMENTING BS/CE 



3.1 Critical Mass 

Change in schools is generally in small increments (and little interventions are easier to ignore). BS/CE was 
an exception, at least in part because the grade-by-grade follow-through strategy concentrated significant 
resources on tightly defined targets. Prior to the 1990-91 inception of BS/CE, the average school owned a 
total of 1 4 computers. If half were then used for administrative/clerical functions and two were in the library, 
that meant that five classrooms might have one (Apple) each. Contrast that with the arrival of four machines, 
a printer, networking capability, software and professional development multiplied by the number of class- 
rooms at that year's target level— all showcased at the beginning of the school year. BS/CE was a big deal, 
especially for the rural schools. 



3.2 Fidelity of Implementation 

BS/CE was also fielded as it was designed. For school improvement in general, the most common explana- 
tion for a lack of outcomes is a lack of implementation— the training is planned but canceled, the books are 
purchased but not required, people are hired for one purpose and end up doing something else. That does 
not describe BS/CE. In the West Virginia case, the districts and the schools did use the discretion that they 
were accorded in software selection and in style of deployment (labs/centers versus distributed versus com- 
bined). But the major points are that not only does BS/CE have effects associated with its implementation, 
it was implemented. 

Our field work included extended visits to 18 schools. The BS/CE computers were where they were sup- 
posed to be (in classrooms or in labs/centers). When teachers were asked to describe how they used the 
BS/CE machines, 80% responded, "reinforcement of the standard curriculum," that is, applications tightly 
targeted on Basic Skills. Although there were some complaints about machines arriving before training or the 
unavailability of technical support, the amount of ’noise' in the West Virginia system was tiny compared to 
the scope and fidelity of the effort. BS/CE was fielded as it was designed. 

BS/CE’s implementation was more influenced by the State than is the usual practice. True, the initial design 
was done by a group that included teacher representation but it is also the case that teachers were required 
to be trained (and supported with salaries or stipends for that). True, counties could choose their software 
suppliers but only between IBM and Jostens and all hardware was IBM. In the final analysis, the reality of 
BS/CE came down to the choices of teachers behind closed classroom doors, but those classroom deci- 
sions were systematically channeled, encouraged and supported in very particular directions. 

3.3 Labs/Centers versus Classroom Distribution 

Schools could choose how to deploy the BS/CE computers— concentrated in labs (6 of the sampled 
schools, 293 students), concentrated in classrooms (5 schools, 273 students) or a combination of lab- 
plus-classroom distributed (7 schools, 380 students). The deployment choice is significant for policy and 
crucial to teachers. (See Figure 5.) 
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Figure 5. BS/CE Computer Doploymont 




Students who had access to BS/CE computers In their classrooms (the "distributed" pattern) did signifi- 
cantly better than students who were taught with BS/CE equipment In lab settings. The students taught In 
the classroom pattern had higher gains In overall scores and In math. They also scored higher on the 1998 
tests than did those In labs. (There were no differences on scores by source of software.) In distributed class- 
rooms, the BS/CE efforts account for 1 9% of the variance In test scores. 

Teachers who had computers m the classroom reported higher skill levels In managing instruction, planning 
lessons, delivering Instruction, and word processing. 

The power of classroom Integration belles the low student:computer ratios reported by teachers In schools that 
used labs as centers. There. 77% of the teachers reported 1 :1 studentcomputer ratios but apparently omitted 
to notice that this only applies during those minutes per week that their students are assigned to the lab. 

Teachers who had BS/CE computers In their classrooms (the 'distributed' pattern) reported more time using 
BS/CE computers for reading, math and writing instruction than either of the other two patterns. Slxty-one 
percent of the teachers with access to computers In their own classrooms said they were confident In using 
computers In their teaching compared to only 43% of the teachers who took their children to the lab for 
Instruction In, about, or with computers. 

4.0 ADDITIONAL EXPLANATIONS FOR TEST SCORE GAINS 

West Virginia's experience with Instructional technology Is Important because of the gains that can be asso- 
ciated with BS/CE. But the State was also changing other aspects of schooling at the same time, in 
understanding the significance of Instructional technology, It is important to Inquire into those other Initiatives. 
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During this period, the State spent $430 million to renovate 470 school buildings. Sixty-eight new schools 
were built. (The state also closed 330 schools in six years, down to 840 buildings.) It is not likely that the 
newness of the physical plant directly impacts learning (Armor s research indicates a slight negative relation). 
But school attendance went up and drop-outs went down in the 1991-1997 period. It may be that a more 
attractive place attracts more attendance and thus increases exposure to teaching and the possibility of 
increased learning. 

In 1989, the year the State legislature passed the BS/CE enabling legislation, West Virginia teachers were 
the 49th worst paid in America. In 1990, there was a statewide strike of teachers. Beginning that year, all 
teachers received a $5,000 pay increase 13 spread over three years: West Virginia's statewide average 
teacher salaries have risen 1 5 places among the states. 

When asked to rank order three phenomena in terms of their probable effect on student learning, 48% of the 
teachers assigned Number One to ‘'technology" followed by “physical school improvements and teacher 
salary increases." The teachers we studied rejected “better salaries « better teaching as too simple an 
explanation for their professional enactment. It is worth noting that the achievement score gains have 
continued although the planned three year schedule of pay increases has long since been completed. In 
general, the literature on teacher compensation suggests that salary is important at two points In the career, 
deciding to get in and deciding to get out. But, in between, what teachers teach and how they teach it is not 
influenced by money. It can be influenced by things like standards, testing, supervision, peer influence and 
instructional technology. 

Whether or not plant renovations and teacher salaries are linked to gain scores, there are compelling rea- 
sons like physical safety and the stability of the teaching force to support improvement in those things. 

Beginning in 1995, West Virginia Bell Atlantic connected 700 of the State's 840 school buildings to Bell 
Atlantic's “World School" Internet service. The presence of all ‘other technology related initiatives' in the 
school is related to achievement score gains but accounts for 0.4% of the difference. Those other initiatives 
included computers purchased from other sources, special grant programs and, in many schools, the “World 
School" Internet service. That capability has very probably helped, but 56% of the teachers acknowledged 
that they knew very little about networking and online communications and only a third of the BS/CE 
teachers have classroom access to the Internet. 

During the years iri which BS/CE has operated, the State legislature has also required local school councils, 
faculty senates, school-site accreditation visits and a “probation" procedure. Those activities, along with the 
requirement that each school have a “Unified School Improvement Plan," are similar to what other states 
have adopted. The policies are reasonable and even wholesome but they have not been unambiguously 
connected to Increases in statewide test scores. There are no states (including West Virginia) in which 
governance modifications have been studied and have been associated with test score gains. 

We have no doubt that changes in school policy, In addition to BS/CE, made some difference. The case of 
“standards" is instructive. The State Department of Education promulgated the “West Virginia Instructional 
Goals and Objectives." In order to win state contracts, both IBM and Jostens Learning correlated their soft- 
ware to that state framework. Once the choice between software vendors had been made, those correlation 

13 Slale taxes were raised $200 million in 1989 and another $200 million in 1990. 
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matrices became a resource if not a guide for Instruction by teachers. And the teachers came to rely on them 
as they integrated BS/CE computers into their teaching. Simultaneously, the State reminded teachers to 
reinforce student basic skills acquisition through the "Teach/Re-Teach" program. The quantitative and the 
qualitative data both support the idea that teachers converged on using BS/CE computers to extend their 
Basic Skills instruction In order to position their students for the related achievement tests. Thus, the 
standards/assessment policies reinforced the policy of technology use Integrated into the curriculum and 
vice versa. 

Our data suggest what expert judgement supports — various policy interventions make various amounts of 
difference and they most probably Interact, 

But, we do not have direct evidence about that, In fact, few changes in schooling depend on demonstrated 
improvements in test scores. Text books get adopted, professional development seminars get scheduled, 
whole theories of learning or of school organization get implemented without any evidence that they impact 
test scores. That does not mean that those phenomena are not worthy or that they might not have an effect 
on student achievement, only that those effects— lack of effects —are not taken into account when decisions 
about such Initiative are made. BS/CE has what others do not — measured results. 



5.0 CONCLUSIONS 

"We need information to show what works and what doesn't. If we had empirical data, policy- 
makers would be more willing to fund technology and voters would be much more wiling to pay." 

- Lieutenant Governor Kim Robak, Nebraska (Kerry White "A Matter of Policy," Education Week, 
November 10, 1997, v. XVII, n. 11, "Technology Counts: Schools and Reform in the Information Age," p 6.) 

The West Virginia legislature funded the Basic Skills/Computer Education program In hopes of levering 
improvements, statewide, in the school achievement of children. On the evidence of the State's change in 
rank relative to other states, and on the evidence of this analysis, the state's expectations were met. 

And West Virginia realized additional outcomes from technology. The schools were able to try out new 
productivity tools, public attitudes toward schools were probably improved. Sixty-two percent of the 
American workforce is already "knowledge workers," people who focus on creating, organizing and com- 
municating Information 14 . BS/CE is a major part of positioning the State's children for that future. 

The improvements reported here were powered by successive waves of hardware and software, the earliest 
generation of which reached back to the late 1980s. At BS/CE's Inception, 486-megahercz machines were 
still on the horizon and the capital Investments necessary to support broadcast-quality audio and video were 
still in the publishers' business plans, If we accept that these gains are a function of that previous generation 
of instructional technology, what will be possible next? 

Ail jurisdictions have an obligation to reach their own conclusions about what is worth doing. The data sug- 
gest the power of instructional technology but they also signal aspects of policy strategy and tactics. 

BS/CE is scalable. The expenditure totals are not out of line with other states. (See also Lewis Solmons 
"Afterword.") The program's components have been well documented by the State. The outcomes are established. 

M CEO Forum on Education and Technology, School Technology and Readiness Report: From Pillars to Progress, Washington DC., October 9, 1997, p 3. The figure 
refers to the 1994 American labor force. 
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Consider the logic of West Virginia's follow-through strategy. Without the yearly concentration of hardware, 
software and training grade-by-grade, no grade would have had a critical mass. A one or two year technolo- 
gy-rich effort would quickly get washed out by children falling back to successive years of technology- poor 
experiences. The performance of the BS/CE lead cohort has distinguished itself and the state. A follow-on 
program called "SUCCESS" Is designed to carry the same equipment and procedures through to the upper 
grades. There Is reason to believe that as BS/CE helped children, so will "SUCCESS; and there Is no 
reason to believe that there will be less technology In the lives of West Virginia's elementary and secondary 
students and graduates. All of that suggests that maintaining this focus will maintain these positive outcomes. 

As other jurisdictions consider instructional technology as an agent of Improvement, Is it of Interest that a 
defined, affordable package of hardware/software/process innovations can account for a large fraction of the 
test score improvement that is available to public policy intervention? And is it of further interest that, In 
addition to test score gains, those Innovations can help position children for a technologically demanding 
economy, society and polity? 

But, BS/CE also has a number of features that are uncommon in the state education policy landscape. The 
features that deliver an installed base critical mass depart from the norm of a small number of computers 
equitably distributed across a large number of classrooms. The choice of software from a fixed set of two 
vendors departs from the conventional ceding of choice among hundreds of vendors to hundreds of schools 
(and often, to thousands of teachers). We believe that part of the explanation for BS/CE's success is the 
defined focus of its implementation. It is worth noting that we did not encounter principals who complained 
that their traditional locc-. autonomy was restricted and neither Is there evidence of anything other than impa- 
tience as the upper grades teachers anticipated their turn on the State's hardware Implementation schedule. 

Still, policy choices, political choices always honor local values, in American schooling, thousands of 
jurisdictions make their own choices. It may be that this documentation of the student outcomes associated 
with West Virginia's program of Basic Skills/Computer Education will advance the consideration of similar 
initiatives In other jurisdictions. 
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Introduction 

The Basic Skills/Computer Education Program (BS/CE) in West Virginia has been one of the most comprehensive 
(in terms of students covered in a state) and long-lived statewide education technology programs ever tried. It is one 
of the few programs that has been in existence long enough to provide answers to the fundamental question of 
whether an infusion of technology tied to the curriculum and associated professional development for teachers affect 
student learning as measured by improvement in scores on tests of basic skills. 

Thus, the Milken Exchange commissioned Interactive, Inc’, to gather information on how the technology was used in 
West Virginia's schools and to determine if technology had any impact on the improvement in the test scores of 
West Virginia's students. This monograph is the report of the results of that study, The principal finding is that 
BS/CE worked. 

Mann, Shakeshaft, Becker, and Kottkamp make this case by pointing out that about 11 percent of the total variance 
(R J - .11 and adjusted R*» .094) in the basic skills achievement gain scores of the 5th-grade students in their sample 
who have had BS/CE since 1991-92 can be explained by a model composed of factors that describe the overall BS/CE 
experience. In other words, about 11 percent of the gain score increase of 5th-grade students can be attributed to 
their participation in BS/CE. Moreover, according to the authors, since about 70 percent of the variation in test scores 
relates to family background, only 30 percent remains that schools can influence. Mann, Shakeshaft, Becker, and 
Kottkamp conclude that the 11 percent of the total variance in the basic skills achievement gain scores (from 1995- 
96 to 199.6-97) can be explained by participation in BS/CE which is about one-third of the 30 percent variance 
remaining after taking into account family-related influences. 

Comparing Technology with Other Policy Initiatives 

These are significant findings and important evidence supporting the claims made by advocates of putting tech- 
nology into all public schools and having it used properly 2 . But in order to recommend similar but more modern 
policies in the future it is not enough to know that BS/CE is related to test score gains; rather, we must be convinced 
that such policies are at least as effective as others that are of similar cost. Thus, to put Mann, Shakeshaft, Becker, 
and Kottkamp's findings in a context of related research, it is useful to look at effect sizes. These can be derived by 
dividing the increase in test scores associated with the regression coefficient for the intervention by the standard 
deviation of test scores in the sample. The regression coefficients and the standard deviations can be found in Mann, 



1 Interactive, Inc. Is o firm whose prlnclpols, Dis. Dolt Mann and Owiol Shakeshaft or# olso professors ol Teochers College, Columbia University ond Hofslra University respectively. 

2 Some people believe that BS/CE Is on application ol old technology thol should not be replicoted. They call it "drill ond proetke" or on Inslructlonol learning system. In fad, BS/CE 
required new designs to mitt West Virginia's needs, and did meet those needs. Although constructivist teaching ond learning have merit in mony situations, o variety of other oppllcctions 
of modern technology moy be appropriate depending upon the need. In studying West Virginia's initiative, it Is inevitable that Interodive, Inc. onalyied the effects associated 
with computer-related technology available beginning in the early 1990s. A decode is a long lime In computer evolution. Observing that BS/CE mode o difference for children Is not the 
same thing os endorsing the current or future opplkotkm ol ten-year old hardware ond software. The size of the effects documented for West Virginia is interesting especially because of 
the generation of hardwote ond software studied. Constructivist ond higher order thinking skills applkotions, when added to the drill ond practice, may well have on even greoter effect. 
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Shskeshaft, Becker, and Kottkamp’s analysis’. The effectiveness of an intervention is viewed as the increase In stan- 
dard deviation units of test scores associated with the intervention. This approach permits comparisons of the effects 
of the BS/CE with other attempts to improve instruction such as class size reduction, tutoring, and the like. 

Most effect size calculations begin with regression analyses In which the educational Intervention is 
measured by a single variable-cither the intervention occurred or did not. The regression coefficient for that dummy 
variable" divided by the standard deviation of the outcome variable gives us a straightforward measure of the effect 
size. Table 1 provides a summary of some effect size studies developed by Benjamin Bloom (in Educational Researcher, 
June/July, 19B4) who cites Walbcrg (1984). 
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Levin, Glass, and Meister (1987) provide estimates of annual effect sizes for four educational interventions (Table 2). 
The effects vary by subject matter and by grade level, and of course, by the specific intervention. For example, the 
mean effect size for cross-age tutoring was .79 for a combined peer and adult program, .97 for the peer 
component and .67 for the adult component The range was 1.02 for second-grade math to .35 for sixth-grade read- 
ing. The mean effect size for reducing class size from 35 to 30 was .06 for math and .03 for reading; reducing class 
size from 35 to 20 produced an effect size of .22 for math and .11 for reading. Adding an additional 30 minutes per 
day to instructional time had a mean effect size of .03 for math and .07 for reading. Finally, a ten-minute daily 
session on a mini-computer for computer-assisted instruction had a mean size effect of .12 for math and .23 for read- 
ing. Each standard deviation is approximately equal to gains of an academic year of 10 months, so each tenth of 
a standard deviation can be viewed as about 1 month of achievement gain per year of intervention (Levin ct al, 1987). 
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.79 


.68 


.55 


.38* 






.42 


.39 


.35 


Reducbni Cl mm Sim 


























From 35 to 30 


.06 












.03 












From 30 to 25 


.07 












.04 












From 25 to 20 


.09 












.05 












From 35*20 


.22 












.11 












inarming Inefcrwctlwml 
Time (eddibonel 30 rrrinuUw 
dot dev for each subject] 


i 

.03 


.02 






.04 




,07 


.08 






.07 





Nota: * - average for grades 8 and 3; * - average for grades 4, 5, and S; * - avsrag a for trades 8 through. 6,. 
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The Mann, Shakeshaft, Becker, and Kottkamp study differs from the ones referred to above in that the BS/CE inter- 
vention is measured not by a single independent variable, but by a set of nine components of that program (grouped 
into three categories), some derived from a factor analysis of responses to several survey questions. In Table 3 we 
divided the regression coefficients on each factor representation by the standard deviation of the test score outcome 
variable to get an "effect size." The quotation marks are used because these effect sizes are unusual ones-not the 
effect of an Intervention as compared to no intervention, but rather the effect of more of the factor versus less. 
Moreover, regression coefficients' size depends on units of the factor, but we do not know what the units of each 
factor are. As an alternative, we provide the possible ranges of each variable as it is constructed to help with the inter- 
pretation. Assuming that the factor analysis converts each factor into comparable units, the effect size measures tell 
us the effect of having more versus less and allow us to compare effects of various aspects of the program. Most of 
the independent variables are scales within a relatively narrow range where a higher score means more of that 
attitude or input 



Calculation of Effact Stow 







SbMferi 




II— iitiitil 


Hanoi •( 


RtflmitiM 


Bwiatai 


Effect 


lata 




gaajfclKl 


late MM 

fcfMIl 


Sot 





SifptflCWId 



Computer and BS/CE \m\ 
time sptnt on BS/CE and 
dma gpant usina other software 


3-15 


1.453 


16.240 


0.069 


0.240 


0.000 


Software access: choice of vendor; 
use of BS/CE specific software 


1-2 


5.953 


16.240 


0.367 


0,184 


0.000 


Computer access: 

■uniinhiiitM of comDuters to students 


1-2 


2.126 


16.240 


0.131 


0.068 


0.115 


Other technology access: 


04 


0.715 


16.240 


0.044 


0.051 


0.278 




Student attitudes 

hncvarflii mm outer* 


24 


2.742 


16.240 


0.169 


0.099 


0.025 


Teacher attitudes 

towards computers: teacher wiUingneea to 
continue learning about computers 
Anri tnchnoloav 


2-10 


1.06S 


16.240 


0.068 


0.123 


0.014 


Teacher confidence: 
teecher comfort level end 

MVerufanna In flllnn mHiniftAPfl 


840 


0.310 


16.240 


0.019 


0.030 


C159B 






Involvement: in the planning 
and implementation of BS/CE 


5-1 


-1.059 


16.240 


-0.121 


-0.097 


0.065 


Tramlng; amount of time teacher* 

tpont teeming on the computer on thair own 


1-5 


1.108 


16.240 


9P68 


0,085 


0.091 
















Sum of al effect sizes 








1.074 







. ' • r 

'46 •* 
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